Input:
	Background

The establishment of DOE brought together a collection of agencies with diverse institutional cultures, structures, and procedures. Since its inception, funding priorities for the department’s varied mission responsibilities have shifted and new challenges have been added. Over the years, DOE’s ability to effectively fulfill these responsibilities has been repeatedly questioned, with calls for dismantling the department reaching a highpoint in the mid-1990s. We concluded at the time that the Congress and the administration needed to rethink DOE’s missions and structure.

		Missions and Organization of DOE

To foster a secure and reliable energy system that is environmentally and economically sustainable; to be a responsible steward of the Nation’s nuclear weapons; to clean up the department’s facilities; to lead in the physical sciences and advance the biological, environmental, and computational sciences; and to provide premier scientific instruments for the Nation’s research enterprise.
DOE groups these responsibilities into four “business lines,” which DOE describes as follows:   Energy resources promotes the development and deployment of systems and practices that provide energy that is clean, efficient, reasonably priced, and reliable;   National nuclear security enhances national security through military application of nuclear technology and by reducing global danger from the potential spread of weapons of mass destruction;   Environmental quality cleans up the legacy of nuclear weapons and nuclear research activities, safely managing nuclear materials, and disposing of radioactive wastes; and   Science advances tools to provide the foundation for the department’s applied missions and to provide remarkable insights into the physical and biological world.
Supporting these mission-related business lines is a “corporate management” function that constitutes a fifth “business line.” This function includes putting in place an effective organizational structure; efficient management practices and information systems; procedures to ensure the safety and health of the department’s workforce and the public, and to protect the environment; and practices to ensure accountability to the public. According to DOE, “the department’s success within its diverse portfolio of programs is largely dependent upon a strong and sound corporate management function.”
DOE’s budget priorities have gradually shifted over the years from energy policy to defense and now environmental cleanup. In fiscal year 2000, the environmental quality business line was the department’s largest budget category, accounting for approximately 34 percent (about $6.7 billion) of its $19.7 billion budget. National nuclear security follows, with 25 percent of the budget (about $5 billion). Science is allotted 16 percent of the budget (about $3.2 billion), and energy resources, the original responsibility of the department, accounts for 13 percent of the budget (about $2.5 billion).
DOE has a workforce of almost 16,000 employees and over 100,000 contractor staff located at over 50 major installations in 35 states. Crucial to DOE’s missions and performance are its 22 laboratories, 11 of which are responsible for multiple programs. Although each of these 11 multiprogram laboratories conducts work in every DOE business line,  3 concentrate on national security issues, 5 on basic science, 2 on environment, and 1 on energy. DOE’s other laboratories are program- specific. The budgets for all 22 laboratories total nearly $8 billion annually.
DOE has a complex structure to manage its diverse missions. All staff and support offices at headquarters report to the Secretary of Energy and a deputy secretary, who serves as the chief operating officer. Below them are two under secretaries: one for national nuclear security, who is also the Administrator of the National Nuclear Security Administration (NNSA), and the other for the energy, science, and environmental missions. A variety of deputy administrators, directors, and assistant secretaries are subordinate to the two under secretaries and oversee individual program areas. DOE has an extensive set of field offices, which are responsible for overseeing contractor performance. The field offices include 11 “operations” offices and several smaller, affiliated “area” and “site” offices, which are usually located at contractor sites. For example, DOE has an area office in the Los Alamos National Laboratory that reports to an operations office in Albuquerque, New Mexico. DOE also has other field offices affiliated with the energy resources business line.
Contractors manage and operate DOE’s facilities and sites under the supervision of department employees. Given that DOE spends most of its budget through these contractors, the ability of DOE to direct, oversee, and hold accountable its contractors is crucial for its mission success and overall effectiveness. DOE’s contracting practices are rooted in the development of the atomic bomb under the Manhattan Project during World War II. Special contracting arrangements were developed by DOE’s predecessor agencies, with participating industry and academic organizations, to reimburse all of the contractors’ costs and to indemnify contractors against any liability they might incur. Most of the current contractors are for-profit companies that receive incentives for meeting certain performance objectives. Several large contractors, however, are nonprofit institutions, such as the University of California, which typically operate research institutions for DOE. Some of these nonprofit contractors also have financial incentives for achieving certain DOE goals.

		GAO’s Call for a New Assessment of DOE

In August 1995 we reported that a fundamental reevaluation of DOE was warranted, based on prior reviews by us, DOE’s Inspector General and other experts, and our survey of experts. All of these reviews identified serious management weaknesses at the department. Our report was neither the first nor the last to recommend rethinking the department’s structure and mission responsibilities.
Our August 1995 report said that DOE had gone through many evolutionary changes since its creation, in part resulting from shifts in priority among its diverse responsibilities. We concluded that even though the department had embarked on some major restructuring, in line with government-wide initiatives to reduce the federal workforce and become more results-oriented, there was no assurance that these reforms would fundamentally alter and improve the ways that DOE managed its missions. We noted that attempting to resolve management weaknesses without first evaluating and achieving consensus on missions was a risky approach to restructuring the department.
Overwhelmingly, our survey of experts concurred that DOE must change. While there was general consensus that DOE should retain and concentrate on essential energy activities, opinions differed on where to place other departmental responsibilities. Most experts considered moving the weapons-related and environmental cleanup responsibilities to other federal agencies and creating a new organizational structure for the national laboratories, such as sharing them among federal agencies or, in some cases, privatizing them. We concluded that the ultimate structure of each mission should be determined by the option that encouraged the most cost-effective practices, attracted necessary technical talent, provided ample flexibility to react to changing conditions, and exhibited the highest degree of accountability.

	DOE Initiated Major Reforms in the 1990s

In the early to mid-1990s, newly appointed Energy Secretary Hazel O’Leary initiated many reforms to address long-standing criticisms of how DOE conducted its business. As part of this process, DOE commissioned various study groups and panels to make recommendations intended to fundamentally improve the department’s efficiency and effectiveness. Based on these recommendations, DOE launched a series of reforms to realign and downsize the agency, as well as address structural weaknesses and improve its management and oversight of contractors. Many of these reforms achieved their immediate objectives.
In 1993, DOE launched an internal initiative to improve safety and awareness of good practices throughout all aspects of the department’s work. The initiative included more attention to risk reduction, improving the qualifications of the workforce, organizational realignment, and moving to external regulation of facilities. In particular, outside reviewers and DOE’s own senior managers questioned the continued justification for the department’s self-regulation of its contractor operated facilities, given that virtually all other federal facilities are externally regulated (including some DOE facilities). In 1994, while legislation was proposed and the Congress held hearings to assess the proposal to move to external regulation, no action was taken. A year later, a DOE advisory committee concluded that secrecy had been used as a shield to deflect public scrutiny of safety and health problems at these facilities, and that the widespread environmental contamination at some facilities was clear evidence that self-regulation had failed.
Also in 1993, the Energy Secretary told the Congress that DOE was not adequately in control of its major facility and site contracts and, therefore, “not in a position to ensure effective and efficient expenditures of taxpayer dollars.” To improve this condition, the Secretary created the Contract Reform Team. (We had previously designated DOE contracting practices as high risk, making the department vulnerable to waste, fraud, abuse, and mismanagement. It remains on our high risk list today.) DOE’s contract reform team made more than 45 recommendations, including a call for strengthening financial information systems, using performance- based contracts, and including performance criteria and incentives in contracts. One significant recommendation urged DOE to shift from making noncompetitive contract awards to adopting a full and open competitive process.
DOE also commissioned two special task forces in 1993 to examine the quality and effectiveness of the department’s laboratories and the management of its energy research and development (R&D) mission. The Secretary of Energy Advisory Board chartered The Task Force on Alternative Futures for the Department of Energy National Laboratories, chaired by a former chairman of the Motorola Corporation, Robert Galvin, to look at the laboratories. The task force’s final report, issued in February 1995, concluded that DOE’s laboratories were in “serious jeopardy, owing to patterns of management and organization that have grown in complexity, cost, and intrusiveness over a long period.”  The report called for a more disciplined research focus by the national laboratories and recommended improvements in DOE management of these facilities, including moving to an independent management structure resembling a government corporation. In response, DOE created the Laboratory Operations Board, an advisory group whose purpose was to provide dedicated management attention to laboratory issues.
The Secretary chartered The Task Force on Strategic Energy Research and Development, chaired by energy analyst Daniel Yergin, to examine DOE’s energy resources business line. The June 1995 report of this task force assessed the rationale for the federal government’s support of energy R&D, reviewed the priorities and management of the overall program, and recommended ways to make it more efficient and effective. The task force recommended that DOE streamline its R&D management, develop a strategic plan for energy R&D, eliminate duplicative laboratory programs and research projects, and reorganize and consolidate the many dispersed R&D programs at DOE laboratories.
The Galvin and Yergin reports led to many changes in how DOE interacts with its contractors, including a streamlining of departmental orders and procedures.
In addition to these improvement efforts, DOE also established a strategic alignment initiative in the fall of 1994, following the results of its extensive strategic planning process. The strategic plan was developed based on the principles of “total quality management” and the desire to increase “stakeholder” participation in decision-making. Under this plan, the department organized itself by “business lines” that were essentially the same as they are today. The first phase of the strategic alignment initiative was employee driven and aimed to identify better, more cost-effective means of performing the core missions of the department as defined in the strategic plan. In May 1995, DOE announced its plan to achieve $1.7 billion in savings over the next 5 years by reducing overhead costs; closing or consolidating field offices; realigning the organizational structure; reducing federal employment; and initiating the delegation of some departmental responsibilities to the private sector (referred to as “privatization”). A portion of the overhead cost savings was to come from externally regulating environment, safety, and health activities; reforming contracting practices; and streamlining departmental oversight. In August 1995, DOE released the specifics of 45 implementation plans, developed in the second phase of the initiative, to guide the cost-saving efforts and improve the department’s performance and accountability.
DOE officials were well aware of the criticism aimed at their department in the early 1990s. While maintaining that their own initiatives could transform the department, DOE officials also recognized that others were calling for more radical changes, ranging from organizing the national laboratories under a corporate structure to completely dismantling the department. DOE officials stated in response to our August 1995 report that while there is “no assurance DOE’s initiatives will succeed, we know that no alternative approach can provide that assurance either.” The department continued to assert that its reforms, unprecedented in its history, would transform the department into a “positive model of organizational change and effectiveness.” According to the Deputy Secretary at the time, the department’s initiative promised to “fundamentally alter how we  look and how we conduct business….”

	Unresolved Management Weaknesses Contribute to Performance Problems

Unresolved management weaknesses have led to recurring performance problems within DOE. Our analysis of more than 200 audit and consultant reports issued since 1995 that pertain to the department identified persistent weaknesses in the integration of strategic plans and information systems; clarification of the respective roles and responsibilities between headquarters and field offices; maintenance of a technically qualified workforce; and implementation of contract management reforms. While many of DOE’s reforms have achieved their immediate objectives, weaknesses persist and have been linked to wide-ranging performance problems, including major cost overruns and schedule delays in a variety of noteworthy projects.

		Strategic Plan Not Used to Organize and Integrate Diverse Missions

DOE has steadily improved its strategic and annual performance plans in response to past criticism. However, the department has not been able to use its strategic plan and other corporate management tools, such as a department-wide information system, to organize and integrate its missions. According to DOE, its strategic plan is a composite of plans guiding the activities of its major programs within the four business lines. This approach has created some management problems that have been identified in our past reports, in particular:   Disconnects exist between the current strategic “business lines” and the way the department is actually organized. While DOE’s strategic goals and objectives are stated within the context of the business lines, the department is organized and managed by its multiple programs. In some cases, several programs contribute to the same business line without any apparent integration. While we have called on DOE to rectify this misalignment, it has not done so. DOE has asserted that its structure is affected by external factors and that no single alignment will yield an organization that eliminates crosscutting objectives. DOE told us that it has therefore organized itself around budget decision units and set program performance measures that are linked to each strategic plan business line.
Shortcomings persist in program planning and priority setting, as well as in the use of strategic goals and measures to describe specific activities. For example, we could not determine from DOE’s 1999 and 2000 accountability and performance reports what the department was trying to accomplish. We also noted that DOE had not corrected the problems in its strategic goals and measures that we identified 2 years ago. According to DOE, changes were made in the FY 2001 Annual Performance Plans to track accomplishments by budget decision units rather than the strategic plan.
DOE has not been able to develop a single strategic plan that integrates its vast laboratory network. The laboratories, particularly the multiprogram ones, operate largely as separate entities. DOE has no central program control over the laboratories, but has instead required that each report to a lead headquarters program office since 1999. Integration into the strategic plan is supposed to occur through the interests of the headquarters offices, even though the major laboratories conduct work in all business lines.   DOE does not have an integrating management information system to consolidate its business, organizational, and operational information throughout the department. In the absence of such an integrating system, mission and program areas have developed their own systems and procedures. A September 2000 DOE Office of Inspector General report noted that duplicative systems existed or were under development at virtually all organizational levels within the department. DOE has acknowledged that a significant barrier to greater departmental integration of information systems has been the Chief Information Officer’s lack of control and influence over the program budgeting processes.
Problems continue with the validity and verifiability of the data used by the information systems to provide a baseline from which to track performance across many parts of the department.

		Roles and Responsibilities Remain Unclear

Since 1995, there have been a number of attempts to clarify roles and responsibilities between headquarters and field staffs to improve lines of authority and accountability. A resolution for this management issue has been elusive because of the way DOE oversees its contractors. Typically, field office managers sign contracts and rate contractors on their performance, but direction on programs or project work comes from the headquarters program offices. Additionally, at least in the past, headquarters staff offices have been allowed to give direct orders to field offices outside of the formal chain of command. The reports that we reviewed frequently cited problems with such intermingled roles and responsibilities.
A 1997 study by the Institute for Defense Analyses revealed that the coordination between DOE programs is an “undisciplined, uncoordinated, essentially ad hoc process between the field managers and each of the program assistant secretaries.” The institute concluded that there was no assurance that resource decisions are weighed against each other in a complete and consistent manner.
A 1999 Panel to Assess the Reliability, Safety and Security of the United States Nuclear Stockpile reported that DOE suffered from a diffusion of functional responsibilities across a range of staff and line organizations that has led to clouded lines of authority and blurred responsibilities and accountability. In 1999, the President’s Foreign Intelligence Advisory Board reported that DOE’s “decentralized structure, confusing matrix of cross cutting and overlapping management, and shoddy record of accountability has advanced scientific and technological progress, but at the cost of an abominable record of security.…” The board labeled DOE’s organization as a “dysfunctional” structure that has too often resulted in mismanagement of security in weapons-related activities and in a lack of emphasis on counterintelligence. The board concluded that “for the past two decades, the Department of Energy had embodied science at its best and security at its worst.”
A 1999 National Research Council review of DOE’s project management problems found that DOE’s “organizational structure makes it much more difficult to carry out projects than in comparable private and public sector organizations.” The council noted that by operating as an aggregate of independent agencies amid various program and field operations offices, DOE had failed to benefit from economies of scale. In 1999 and 2000, we attributed problems at DOE’s Spallation Neutron Source project under construction in Oak Ridge, Tennessee, and at DOE’s National Ignition Facility being built in Livermore, California, to, among other problems, DOE’s complex management and organizational structure and unclear lines of authority.
A March 2000 National Academy of Public Administration report on DOE’s Energy Efficiency and Renewable Energy Office found that the office had suffered from unclear roles and responsibilities among various organizational levels. The Academy noted that there are “significant differences in  understanding of the roles and responsibilities for program and project management.”
Recognizing these problems, DOE has changed reporting relationships between headquarters and field offices in an attempt to clarify lines of authority and to strengthen accountability. The latest major realignment occurred in 1999 with the assigning of field offices to lead program secretarial offices at headquarters. In addition, a Field Management Council was established to coordinate the direction given to the field by program and support offices. DOE’s field offices now report to whichever headquarters program office provides the most funding to the contractor sites overseen by the field managers—an approach used without success in the past. This realignment had to be modified slightly in late 2000 to accommodate the establishment of NNSA. The current reporting arrangement, however, has given rise to some new management problems. We found, for example, that there is considerable uncertainty about reporting relationships in situations where many different headquarters programs support activities at shared facilities and complexes. This problem is particularly acute at DOE’s multiprogram national security laboratories, where work is conducted on all of DOE’s missions, yet field management must report only to NNSA headquarters. Thus, non-NNSA program staff in headquarters must work through NNSA management in the field to accomplish work related to the science and environmental missions. Conversely, some NNSA staff members work in field offices that report to headquarters programs in science or environmental management, even though they can receive direction only from NNSA. Various memorandums of agreement have been created to sort out these arrangements and to provide support services across business lines. However, staff in some field offices that we visited told us that they are unsure how the new reporting relationships will work.
The establishment of NNSA has yet to clarify roles and responsibilities within the nuclear security business line and may have exacerbated reporting relationships, at least temporarily. In early 2001, we and the Panel to Assess the Reliability, Safety and Security of the United States Nuclear Stockpile challenged NNSA to develop a plan for fundamentally redefining roles and responsibilities among its headquartered and field organizational units. The panel called on NNSA to “clarify functional authority, reduce management layers, eliminate micromanagement [of the laboratories], and downsize.” As late as April 2001, we found that NNSA had not specified the roles and responsibilities of each of the headquarters offices; the relationship between the headquarters and the field offices; whether headquarters or field offices will direct and oversee contractors; and the relationship between the NNSA staff and the rest of DOE. In NNSA’s May 2001 interim report, the administration stated that it intended to seek expert advice on clarifying relationships between headquarters and the field, as well as on other issues in preparation for an October 2001 status report to the Congress. On June 26, 2001, in testimony before the House Armed Services Committee, the chairman of the Panel to Assess the Reliability, Safety and Security of the United States Nuclear Stockpile noted that “some of the more fundamental management problems [with DOE] still remain to be addressed.”

		Lack of Qualified Staff Has Impeded Effective Contractor Oversight

Lack of technically qualified staff within DOE is another long-standing management weakness that has been linked to performance problems. We have raised concerns about this weakness since 1991, and many other external reviewers have echoed these concerns since then. For example, a 1997 report by the Institute for Defense Analyses pointed out deficiencies in the technical capabilities of those DOE managers who had survived departmental downsizing. In addition, the Defense Nuclear Facilities Safety Board warned in 1997 that, given likely future reductions in DOE’s budget, the department needed to make advance preparations to avert the loss of technically competent safety personnel.
Responding to these and other concerns, the department announced a new Workforce for the 21st Century Initiative to strengthen technical and management capabilities for its mission requirements. In particular, a 1998 internal DOE study confirmed the need to develop programs to address workforce management weaknesses in the procurement environment, such as recruitment, retention, and succession planning. However, despite these actions, additional internal and external reports that followed have raised concerns about the qualifications of DOE’s workforce.
We reported in 1999 that while the Spallation Neutron Source project appeared to be on schedule, it had already exhibited warning signs of failure because it lacked personnel with technical skills and managerial experience. In 1999, the Commission on Maintaining United States Nuclear Weapons Expertise found that DOE’s aging workforce, the tight market for talent, the lack of a long-term hiring plan, and other constraints had raised serious doubts that the department would be able to maintain its nuclear weapons expertise in the future. In 1999, the National Research Council found that DOE did not have “the necessary experience, knowledge, skills, procedures or abilities to prepare good performance measures” for its contracts.
In its fiscal year 2001 Annual Performance Plan, the department stated that it had “fully addressed” the lack of technical and management skills by establishing a Corporate Education, Training and Development Plan in fiscal year 1999. DOE pointed out that it had training programs in place for procurement professionals, property managers, and information management specialists, and that it was establishing a new program to rebuild a talented and well-trained corps of R&D technical program managers. In particular, DOE reported in March 2000 that it had initiated a program to develop future leaders of the acquisition workforce. The Defense Nuclear Facilities Safety Board’s 2000 report credited DOE with taking steps to improve the technical capabilities of personnel at its defense nuclear facilities, but pointed out the need for DOE’s leadership to pay increased attention to this issue and to follow through with its improvement plan. Notwithstanding these efforts, the department has now acknowledged that its workforce weaknesses represent a much broader challenge encompassing the larger arena of human capital management.
In commenting on a draft of this report, DOE said it had additional efforts in workforce restructuring. In support, DOE officials provided us with its September 2001, “Five-Year Workforce Restructuring Plan,” prepared in response to an Office of Management and Budget requirement of all federal agencies. The plan describes itself as a “corporate roadmap” for, among other things, reducing manager and organizational layers, increasing spans of control, and redeploying positions.

		Contract Management Reforms Not Fully Implemented

DOE has made process improvements in its contracting by implementing many of the 1994 contract reform team recommendations. For example, DOE has increased competition, imposed greater contractor liability, phased in performance-based incentives, and begun using results-oriented statements of work. According to DOE, 26 of its 37 major site and facility management contracts have now been competed, up from just 3 prior to 1994. All of these new contracts employ performance-based techniques in defining contractor requirements, evaluating performance, and linking financial incentives to results. In addition, according to DOE, there has been an overhaul and standardization of contract regulations and the issuance of guidance on proper contract administration. Nonetheless, the department has been criticized for not fully implementing its contract reforms, as noted in several reports.
In an October 1997 report, DOE’s Inspector General reported problems with performance-based contracting at DOE’s Nevada Operations Office. The report found that performance-measurement milestones had been estimated after the work had actually been completed. In addition, performance measures associated with this aspect of the contract were vague, leading DOE to reward performance that could not be objectively validated. In May 1999, we reported that while DOE laboratory contracts we examined had some performance-based features, there was a wide variance in the number of performance measures and the types of fees negotiated. We also found that DOE had not determined whether giving higher fees to encourage superior performance by laboratory contractors is advantageous to the government.
The National Research Council’s 1999 report concluded that DOE has had limited success in establishing and managing performance-based contracts. In its 2001 follow-up report, the Council noted that DOE has yet to devise and implement either a contract performance measurement system or an information system that can track contracts and contractor performance while cycling information back into key decisions.
DOE’s Inspector General reported in April 2000 that performance-based incentives in the contract for DOE’s Idaho National Engineering and Environmental Laboratory had not been fully successful in improving performance and reducing costs. For some incentives, performance declined or remained unchanged. For other incentives, performance improved, but the gains were overstated, the contractor was compensated twice, improvements either could not be linked directly to actions taken by the contractor during the incentive period or were made for a disproportionately high fee, and the contractor could not demonstrate any reduction in cost.
DOE’s Inspector General has also identified other areas where contract reforms have not been fully implemented, including the following:   A November 1998 audit determined that 16 of DOE’s 20 major for-profit operating contracts did not incorporate liabilities provisions called for under contract reform.
A December 1999 audit concluded that the department’s award procedure “effectively circumvented federal requirements designed to promote and ensure the appropriate use of competition in contracting.”
A January 2000 audit of outsourcing opportunities at the Los Alamos National Laboratory determined that although the laboratory contractor found that only 4 of 184 support services could potentially be obtained at lower cost from outside entities, in fact at least 128 had outsourcing potential.
A February 2000 audit found that only one of the four contractors reviewed had fully met a requirement to prepare “make-or-buy” plans to obtain supplies and services on a least-cost basis.
A January 2000 summary report on management challenges facing DOE pointed out that while incentives have been included in most contracts, reviews show systemic weaknesses in the way these incentives have been administered. Incentive fees have risen dramatically, but there has been no commensurate increase in financial risk to DOE’s major contractors.
DOE has also struggled to effectively implement its privatization program, which is intended to keep the department’s environmental cleanup projects on schedule at budgeted costs. For example, the cleanup contracts were terminated at two noteworthy privatization projects—the Hanford tank-waste project and the Idaho Pit 9 cleanup project—because of concerns with rapidly escalating costs and the contractor performance.
Finally, while DOE has increased the number of major site and facility contracts that it awards competitively, several major contracts have not been, including nine contracts with a combined value of $22 billion. Furthermore, despite glaring performance problems at certain laboratories, DOE has excluded its largest laboratories from full and open competition. For example, DOE’s contracts with the University of California to operate two national laboratories have not been opened to competitive bidding since they were awarded over 50 years ago, despite reported security and project management problems at these laboratories. In commenting on a draft of this report, DOE said that it has not been required to competitively award these types of contracts (Federally Funded Research and Development Centers) and that it “actively considers the use of competitive procedures for such contracts and has competed them where appropriate.” DOE also said that it retained its contracts with the University of California based on “national security considerations.”

		Persistent Management Weaknesses Contribute to Project Management Problems

Several of the unresolved management weaknesses that we identified have been linked to recurring problems with the management of programs and projects. In 1997, we documented that over a 16-year period, of 80 DOE projects started that cost at least $100 million each, only 15 were completed, with most of these experiencing scheduling delays and cost overruns; 31 were terminated; and the 34 ongoing projects were exhibiting scheduling delays and cost increases. Since 1995, DOE and its contractors have drawn a litany of criticism for poor performance on several specific projects, including the following.
DOE projects commonly overrun their budgets and schedules, leading to pressures for cutbacks that have resulted in facilities that do not function as intended, projects that are abandoned before they are completed, or facilities that have been so long delayed that, upon completion, they no longer serve any purpose. In short, DOE’s record calls into question the credibility of its procedures for developing designs and cost estimates and managing projects.
The Council not only reiterated a listing of past project failures, but also noted that 26 major projects under review at the time of its study were showing notable deficiencies in project management. The report concluded that DOE’s prior efforts to solve project management problems had been so unsuccessful that achieving improvements in this area would require fundamental changes in organizational structures, documents, policies and procedures, as well as drastic changes in the “culture” of the department.
DOE acknowledged the persistence of problems in its project management practices in the department’s fiscal year 2001 performance and accountability report. DOE stated that “the results from 33 independent external project reviews, undertaken this past year, indicate serious systemic issues needing correction. Among the most prevalent problems are inadequacies in technical scope, schedule planning and control, cost estimating, and lack of clarity on roles and responsibilities.”
In response to the Council’s 1999 recommendations for improving project management in DOE, the department created the central Office of Engineering and Construction Management and affiliated support offices in the three largest departmental program offices. These offices intend to create new policies and procedures, conduct independent project reviews, and train staff in project management practices. The department also plans to create a career track for project managers. However, a follow-up report by the Council in January 2001 raised concerns about DOE’s leadership commitment to implementing the report’s recommendations, particularly regarding the role of the Office of Engineering and Construction Management.
In commenting on a draft of this report, DOE said that many of its projects are "unique, one-of-a-kind" ventures that contain significant research and development which can impact cost and schedule assumptions. We agree with DOE that its projects are often challenging. We also agree that such challenges are not an excuse for poor project management performance, a common problem in many DOE activities.

	Diverse Missions, Dysfunctional Structure, and Weak Culture of Accountability Are Fundamental Impediments to Improvement

The persistence of DOE management weaknesses and project problems, despite the many actions taken by the department to improve its performance, are indicative of underlying impediments that have not been addressed. We found that the department’s diverse missions, dysfunctional organizational structure, and weak culture of accountability impede fundamental improvement at DOE. Unless these underlying and interrelated impediments are addressed, DOE’s management and performance problems will likely continue.

		Diverse Missions Resist Integration

Fundamental improvement in DOE’s performance is impeded by the difficulty of effectively integrating the management of the department’s diverse missions. DOE’s energy, environmental, science, and national nuclear security staffs operate largely as separate entities within the department, maintaining their own operating styles and decision-making practices. For example, some mission areas retain strong central control over their programmatic actions, as in the science area, while others delegate more of this responsibility to the field, as in the environmental area. Uncoordinated and inconsistent direction from program headquarters offices still places the burden of effectively integrating varying goals, objectives, and management styles on the field managers who must manage this diversity at shared facilities.
The National Research Council’s 1999 report on DOE project management noted that “cultures, attitudes and organizational commitments have shaped service delivery, and as DOE’s missions changed in response to external conditions, the diversity of cultures inherited by the department’s collection of agencies did not necessarily change with it.” This diversity of mission cultures under one roof has long prevented DOE from developing a consistent approach in its systems, structures, and interactions with contractors. For example, DOE’s national security programs have a long history of operating in secret, which leads to practices that are quite different from DOE’s science programs, which are more open and flexible—yet these programs operate at shared facilities. This clustering of diverse programs has complicated lines of authority, thus diluting accountability among staff, and has impeded DOE’s ability to oversee contractors.
It has been difficult for DOE to meet all the priorities of its mission programs and the requirements of the department staff offices. For example, more management attention has sometimes been given to DOE contractors meeting nuclear weapons program goals than to operating safely and in an environmentally responsible manner. The widely publicized security problems at the Los Alamos National Laboratory in 1999 and 2000 are another example. DOE’s contract with Los Alamos contained few incentives for controlling classified material but many rewards for high quality science work—yet this work was taking place in a top-secret laboratory, whose primary mission is designing nuclear weapons. As a result, although laboratory staff performed security tasks poorly, such lapses had limited impact on the lab contractor’s overall DOE rating and subsequent performance fee.
In the future, the task of integrating diverse missions will likely be complicated by the need to place additional emphasis on DOE programs that play a role in ensuring homeland security. Such programs include critical infrastructure protection; nonproliferation programs, which aid in keeping nuclear material and weapons knowledge out of the hands of terrorists; R&D; and emergency preparedness.

		Organizational Structure Precludes Effective Management and Performance

Over the last decade or so, DOE has undertaken major departmental shake-ups every two or three years. None have stemmed recurring fundamental problems and all have been thwarted by institutional intransigence.
The most problematic organizational problems have involved the nuclear weapons complex. Years of tinkering with reporting relationships between the offices that have a role in national nuclear security and the laboratories where most weapons-related work is performed have not yielded many positive results. For example, the Special Investigative Panel of the President’s Foreign Intelligence Advisory Board noted in its 1999 report that “convoluted, confusing, and often contradictory reporting channels have made the relationships between DOE headquarters and the laboratories, in particular, tense, internecine, and chaotic.” In addition, the panel found that much of the confusion centered on the role and power of the field offices. As the panel reported, “senior DOE officials often described these offices as redundant operations that function as shadow headquarters, often using their political clout and large payrolls to push their own agendas and budget priorities in the Congress.”
To address long-standing security problems across the nuclear weapons complex, the panel concluded that because “DOE was incapable of reforming itself—bureaucratically and institutionally—in a lasting way,” an autonomous structure should be established for the national nuclear security business line, free of all other obligations imposed by DOE management. Specifically, the panel recommended creation of a new agency that is far more mission-focused and bureaucratically streamlined. Instead, the semiautonomous NNSA was established within the department.
DOE and NNSA officials are now attempting to develop and implement an organizational plan that can operate effectively within DOE’s overall field and headquarters structure. Historically, DOE’s efforts to reorganize assumed that current missions will be retained under any new structure. However, as DOE’s Laboratory Operations Board concluded in December 2000, the creation of NNSA will present organizational and management challenges, especially maintaining a national laboratory system that can meet the department’s current mission requirements. Making changes in the current environment is further complicated by the need to consider DOE’s potentially expanded role on homeland security matters on overall departmental missions.

		Weak Culture of Accountability

DOE’s lack of a strong culture of accountability is the third basic impediment to improved performance. A number of factors have weakened accountability in the department. DOE’s organizational structure, which has blurred lines of authority, has made it difficult to hold staff and contractors accountable for poor performance. In addition, DOE has not taken action to improve the accountability of the organization in other areas that were identified in the mid-1990s. These pertain to contracting practices, health and safety regulation, and human capital management.
The reluctance of past Secretaries to open all major DOE site and facility contracts to competitive bidding has diluted accountability by weakening the department’s position with its contractors. Only once has DOE fired a contractor for performance problems (at Brookhaven National Laboratory in May 1997), and rarely has it taken aggressive action to hold contractors accountable, even in the face of major project failures.
DOE’s shifting policies on external regulation also reflect DOE leadership’s ambivalence toward accountability. Despite the position of former Secretary O’Leary—and her internal managers and consultants— that external regulation would give DOE credibility and make its facilities safer, subsequent leaders reversed course. At first, Secretary Federico Peña, O’Leary’s successor, slowed the process by ordering a pilot program of external regulation concepts. His cautious approach was meant to test how regulators might treat DOE, and at what cost. His successor, Secretary Bill Richardson, concluded that external regulation was not worth pursuing because the costs would likely outweigh the benefits. However, this position conflicted with DOE’s own pilot program results and was inconsistent with conclusions reached by the Nuclear Regulatory Commission and the Occupational Safety and Health Administration— DOE’s likely regulators.
Finally, DOE’s leadership has not devoted enough attention to recruiting and training a qualified technical workforce, even though these needs have been known for over a decade. Without such staff, the department lacks the expertise to direct and oversee contractors working on highly technical matters and hold them accountable for poor performance.

	Conclusions

Past DOE leadership has not succeeded in transforming the Department into an effective agency, as shown by the persistence of management weaknesses that have led to the performance problems documented in this report. Historically, DOE has made piecemeal changes in response to problems or criticisms without assessing the root causes of its management weaknesses: DOE’s diverse missions, dysfunctional organizational structure, and weak culture of accountability.
While DOE should take immediate steps to strengthen accountability, addressing the impediments to improved performance stemming from its diverse missions and dysfunctional organizational structure will require consultation with the Congress and other federal agencies. Since 1995, legislation has been introduced each year to eliminate DOE and transfer its missions to other agencies, or to terminate some of its R&D programs and laboratories. The establishment of NNSA might suggest opportunities to reconfigure other business lines, as some have suggested for the Office of Science. While the program activities of the department are important, that does not mean that all can be best managed under one agency or that each is inherently governmental.
DOE must also have an organizational structure that effectively meets the needs of the department’s missions. However, given the current diversity of these missions, the semi-autonomous status of the NNSA, and shifting mission emphases, such as protecting energy infrastructure, establishing an optimum structure embracing all of DOE’s missions may simply not be possible. New leadership, ongoing organizational changes, and the need to consider how DOE’s responsibilities contribute to homeland security missions, make this an opportune time to address the root causes of performance problems in DOE.

	Recommendations for Executive Action

To address its diverse mission and organizational issues, we recommend that the Secretary of Energy, in consultation with the Office of Management and Budget and other federal agencies that might gain or lose missions if DOE were reconstructed, develop a strategy for determining whether some missions would be managed better if located elsewhere, combined with other agencies, or privatized. Once this is accomplished, the Secretary should report his findings and a proposal to realign the various missions to the Congress.
Pending the results of a comprehensive review of DOE’s missions, the Secretary of Energy should take immediate steps to improve the department’s accountability. Such steps should include, for example, ensuring that all contract-reform initiatives already under way are completed, holding staff and contractors strictly accountable for performance, ending self regulation of worker and nuclear safety in its facilities, and developing a more technically competent workforce.

	Agency Comments

In commenting on a draft of our report, DOE said that the Secretary "recognizes and accepts" many of our points and has already "instituted a path forward for achieving his vision of excellence."  DOE also noted that its management challenges are "enormous" and efforts to resolve them "will take time." An important effort under way, according to DOE, is its "strategic mission review," for which a report is due in January 2002. According to DOE, the purpose of this review is to focus the department on activities that best support its "overarching national security mission." DOE also listed several other steps that it said will help clarify roles and responsibilities, streamline its organizational structure, and instill stronger accountability among federal and contractor staff.  Further, DOE said it has launched initiatives to "determine why previously identified problems have not been addressed." Finally, the department said that the sum of its ongoing initiatives should enable it to "achieve the spirit" of our recommendations to improve mission, structure, and accountability.
DOE's many initiatives, if fully implemented, address several management challenges that have long plagued the department. However, while it is too early to assess the effectiveness of these initiatives, we are concerned that they may not adequately address the root causes of DOE’s recurring performance problems, particularly those related to the department's diverse missions. For example, while we applaud the Secretary's efforts to provide a strategic focus to guide all program activities, it is unclear how a “national security” mission can subsume each of DOE’s highly diverse programs in science, environmental quality, and energy resources. Developing measurable national security objectives for environmental management, DOE’s largest budget category, will be particularly challenging.
Also, it appears that DOE's "strategic mission review" assumes that each of its many missions is still best managed by the department. As we noted in our report, many of DOE's structure and accountability problems stem from the nearly impossible task of managing diverse (and sometime conflicting cultures) within a common field structure.  The role and responsibility problems that result from this condition will likely persist, absent a comprehensive evaluation of how and where best to manage each mission.  The creation of NNSA was an attempt to resolve some of these issues internally, but the effectiveness of its management structure and associated processes is still highly uncertain.  In particular, DOE has still not clearly defined roles and responsibilities for NNSA’s headquarters and field units or relationships with the rest of the department.  DOE's task of developing an integrated department is made more difficult by an expanding mission emphasis on safeguarding energy infrastructure and enhancing homeland defense against terrorist threats.  We believe that with these new mission emphases and the persistent questions about how NNSA will operate relative to other DOE programs, it is more important than ever for a strategic mission review to focus on determining whether some missions would be managed better if located elsewhere, combined with other agencies, or privatized.  As we explained in our report, a comprehensive mission assessment would require the Secretary to consult with the Office of Management and Budget and other federal agencies that might gain or lose missions if DOE were restructured.
Many of the organizational changes cited by DOE are positive steps, such as clarifying the roles of the deputy and undersecretary, and creating a Field Management Council to facilitate cooperation among the department’s diverse programs.  However, past experience has shown that such process changes have merely tinkered with a flawed structure. Without a serious effort to consider each mission for its proper placement in or out of DOE, the structural problems that have clouded roles and responsibilities will likely persist. Therefore, we reaffirm our recommendation that DOE develop a strategy for realigning its missions, followed by a proposal to the Congress.
Finally, while DOE cited numerous initiatives to strengthen accountability, it is too early to judge whether these and other efforts adequately address our recommendation in this area.  In particular, we note that none of the initiatives cited by DOE would end self-regulation of nuclear and worker safety in its facilities.  Moreover, DOE leadership has not been able to fully implement and sustain past initiatives aimed at improving accountability among federal and contractor staff.
Appendix III includes the full text of DOE's comments and our response.
We conducted our review from November 2000 through September 2001 in accordance with generally accepted government auditing standards. Appendix I provides details about the scope and methodology of our review.
As arranged with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution of it until 15 days from the date of this letter. We will then send copies to the Secretary of Energy; the Director, Office of Management and Budget; appropriate congressional committees; and other interested parties. We will also make copies available to others on request.

Appendix I: Scope and Methodology

We conducted our analysis primarily through an assessment of more than 200 external and internal reviews of the Department of Energy (DOE) since August 1995. We selected this date as a baseline because it coincides with our first call to assess DOE’s structure and missions, based on a series of prior reports on the department. In addition, we relied on information from interviews and internal documents obtained previously from DOE headquarters in Washington, D.C., and operations offices in the field that are affiliated with the three largest program offices. These field offices included the Oakland Operations Office in California, aligned with the National Nuclear Security Administration (NNSA); the Chicago Operations Office in Illinois, aligned with the Office of Science; and the Savannah River Operations Office in South Carolina, aligned with the Office of Environmental Management.
To describe actions taken by DOE to improve its performance by the mid-1990s, we reexamined our 1995 report on a framework for restructuring DOE and its missions. We also reviewed documents pertaining to the reforms initiated by DOE at the time of our report, including the results of several noteworthy task forces that were established by the department. We relied primarily on the department’s comments on our August 1995 report to represent DOE’s position on the significance of its initiated reforms.
To assess DOE progress since the mid-1990s in addressing management weaknesses and improving performance, we searched our database for reviews of DOE that we published between August 1995 and May 2001. Of the more than 225 reports identified, we selected 121 that addressed DOE corporate management functions, including strategic planning; information technologies; retaining, recruiting and training staff; security; environment, safety and health practices; contracting; program and project management; and national laboratory reform. We prepared summaries of the observations and recommendations contained in each of these reports. We chose not to include reports that addressed either independent agencies within the department or issues that do not consume many DOE resources. Specifically, we excluded reports on the Nuclear Regulatory Commission, the Federal Energy Regulatory Commission, the Power Marketing Administration, the Tennessee Valley Authority, and issues related to global climate change. With the exception of our major management challenges reports on DOE, the reports that we included were limited in scope and addressed only specific issues under review. The reports, therefore, do not cover all of the program and project activities of the department. For example, there was limited review of the department’s energy resources business line. To improve our coverage of the department, we searched other sources of reports to identify 87 additional documents that addressed the department’s performance since 1995. The Congressional Research Service, DOE’s Inspector General, the National Research Council, the National Academy of Public Administration, several DOE task forces and commissions, as well as the department, were among those organizations that prepared these reports. Appendix II lists the reports and other documents that we reviewed.
To identify any underlying impediments to more effective management and improved performance at DOE, we reviewed our collection of reports to determine the possible causes behind the recurring management weaknesses. While there was no single source among the reports reviewed that explicitly observed all three of our root causes, there were many documents that mentioned one or two of them as contributing to a departmental culture that resists fundamental change. We assessed the strength and pervasiveness of these root causes, as well as the actions of past DOE leadership, to draw our conclusions and recommendations.
We conducted our review from November 2000 through September 2001 in accordance with generally accepted government auditing standards.

Appendix II: Documents Reviewed

	GAO Reports

Department of Energy: Views on the Progress of the National Nuclear Security Administration in Implementing Title 32 (GAO-01-602T, Apr. 1, 2001).
Information Security: Safeguarding of Data in Excessed Department of Energy Computers (GAO-01-469, Mar. 29, 2001).
Nuclear Cleanup: Progress Made at Rocky Flats, but Closure by 2006 Is Unlikely, and Costs May Increase (GAO-01-284, Feb. 28, 2001).
High Risk Series: An Update (GAO-01-263, Jan. 2001).
Major Management Challenges and Program Risks: Department of Energy (GAO-01-246, Jan. 2001).
Nuclear Weapons: Improved Management Needed to Implement Stockpile Stewardship Program Effectively (GAO-01-48, Dec. 14, 2000).
Financial Management: Billions in Improper Payments Continue to Require Attention (GAO-01-44, Oct. 27, 2000).
Reinventing Government: Status of NPR Recommendations at 10 Federal Agencies (GAO/GGD-00-145, Sept. 21, 2000).
Government Performance and Results Act: Information on Science Issues in the Department of Energy’s Accountability Report for Fiscal Year 1999 and Performance Plans for Fiscal Years 2000 and 2001 (GAO/RCED-00-268R, Aug. 25, 2000).
National Ignition Facility: Management and Oversight Failures Caused Major Cost Overruns and Schedule Delays (GAO/RCED-00-271, Aug. 8, 2000).
Department of Energy: Uncertainties and Management Problems Have Hindered Cleanup at Two Nuclear Waste Sites (GAO/T-RCED-00-248, July 12, 2000).
Nuclear Security: Information on DOE’s Requirements for Protecting and Controlling Classified Documents (GAO/T-RCED-00-247, July 11, 2000).
Observations on the Department of Energy’s Fiscal Year 1999 Accountability Report and Fiscal Year 2000/2001 Performance Plan (GAO/RCED-00-209R, June 30, 2000).
Nuclear Waste Cleanup: DOE’s Cleanup Plan for the Paducah, Kentucky, Site Faces Uncertainties and Excludes Costly Activities (GAO/T-RCED-00-225, June 27, 2000).
Department of Energy: National Security Controls Over Contractors Traveling to Foreign Countries Need Strengthening (GAO/RCED-00-140, June 26, 2000).
Nuclear Waste: Observations on DOE’s Privatization Initiative for Complex Cleanup Projects (GAO/T-RCED-00-215, June 22, 2000).
Information Security: Vulnerabilities in DOE’s Systems for Unclassified Civilian Research (GAO/AIMD-00-140, June 9, 2000).
Nuclear Waste: DOE’s Advanced Mixed Waste Treatment Project: Uncertainties May Affect Performance, Schedule, and Price (GAO/RCED-00-106, Apr. 28, 2000).
Nuclear Waste Cleanup: DOE’s Paducah Plan Faces Uncertainties and Excludes Costly Cleanup Activities (GAO/RCED-00-96, Apr. 28, 2000).
Federal Research: DOE Is Providing Independent Review of the Scientific Merit of Its Research (GAO/RCED-00-109, Apr. 25, 2000).
Low-Level Radioactive Wastes: Department of Energy Has Opportunities to Reduce Disposal Costs (GAO/RCED-00-64, Apr. 12, 2000).
Department of Energy: Views on Proposed Civil Penalties, Security Oversight, and External Safety Regulation Legislation (GAO/T-RCED-00-135, Mar. 22, 2000).
Nuclear Security: Security Issues at DOE and Its Newly Created National Nuclear Security Administration (GAO/T-RCED-00-123, Mar. 14, 2000).
Nuclear Nonproliferation: Limited Progress in Improving Nuclear Material Security in Russia and the Newly Independent States (GAO/RCED/NSIAD-00-82, Mar. 6, 2000).
Department of Energy: Views on DOE’s Plan to Establish the National Nuclear Security Administration (GAO/T-RCED-00-113, Mar. 2, 2000).
Nuclear Security: Improvements Needed in DOE’s Safeguards and Security Oversight (GAO/RCED-00-62, Feb. 24, 2000).
Occupational Safety and Health: Federal Agencies Identified as Promoting Workplace Safety and Health (GAO/HEHS-00-45R, Jan. 31, 2000).
Nuclear Weapons: Challenges Remain for Successful Implementation of DOE’s Tritium Supply Decision (GAO/RCED-00-24, Jan. 2000).
Nuclear Waste: DOE’s Hanford Spent Nuclear Fuel Storage Project— Cost, Schedule, and Management Issues (GAO/RCED-99-267, Sept. 20, 1999).
Department of Energy: Uncertain Future for External Regulation of Worker and Nuclear Facility Safety (GAO/T-RCED-99-269, July 22, 1999).
Observations on the Department of Energy’s Fiscal Year 2000 Performance Plan (GAO/RCED-99-218R, July 20, 1999).
Department of Energy: Problems in the Management and Use of Supercomputers (GAO/T-RCED-99-257, July 14, 1999).
Department of Energy: Need to Address Longstanding Management Weaknesses (GAO/T-RCED-99-255, July 13, 1999).
Nuclear Safety: Department of Energy Should Strengthen Its Enforcement Program (GAO/T-RCED-99-228, June 29, 1999).
Nuclear Weapons: DOE Needs to Improve Oversight of the $5 Billion Strategic Computing Initiative (GAO/RCED-99-195, June 28, 1999).
Department of Energy: DOE’s Nuclear Safety Enforcement Program Should Be Strengthened (GAO/RCED-99-146, June 10, 1999).
Department of Energy: Cost Estimates for the Hanford Tank Waste Remediation Project (GAO/RCED-99-188R, May 19, 1999).
National Laboratories: DOE Needs to Assess the Impact of Using Performance-Based Contracts (GAO/RCED-99-141, May 7, 1999).
Nuclear Waste: DOE’s Accelerated Cleanup Strategy Has Benefits but Faces Uncertainties (GAO/RCED-99-129, Apr. 30, 1999).
Department of Energy: Accelerated Closure of Rocky Flats: Status and Obstacles (GAO/RCED-99-100, Apr. 30, 1999).
Nuclear Waste: Process to Remove Radioactive Waste From Savannah River Tanks Fails to Work (GAO/RCED-99-69, Apr. 30, 1999).
Department of Energy: Key Factors Underlying Security Problems at DOE Facilities (GAO/T-RCED-99-159, Apr. 20, 1999).
DOE Management: Opportunities for Saving Millions in Contractor Travel Costs (GAO/RCED-99-107, Apr. 1, 1999).
Department of Energy: Usefulness of Performance Plan Could Be Improved (GAO/T-RCED-99-134, Mar. 24, 1999).
Department of Energy: Challenges Exist in Managing the Spallation Neutron Source Project (GAO/T-RCED-99-103, Mar. 3, 1999).
Nuclear Nonproliferation: Concerns With DOE’s Efforts to Reduce the Risks Posed by Russia’s Unemployed Weapons Scientists (GAO/RCED-99-54, Feb. 19, 1999).
Department of Energy: Actions Necessary to Improve DOE’s Training Program (GAO/RCED-99-56, Feb. 12, 1999).
Major Management Challenges and Program Risks: Department of Energy (GAO/OGC-99-6, Jan. 1999).
Nuclear Weapons: Key Nuclear Weapons Component Issues Are Unresolved (GAO/RCED-99-1, Nov. 9, 1998).
Department of Energy: Management of Excess Property (GAO/RCED-99-3, Nov. 4, 1998).
Department of Energy: DOE Needs to Improve Controls Over Foreign Visitors to Its Weapons Laboratories (GAO/T-RCED-99-28, Oct. 14, 1998).
Nuclear Waste: Department of Energy’s Hanford Tank Waste Project— Schedule, Cost, and Management Issues (GAO/RCED-99-13, Oct. 8, 1998).
Nuclear Waste: Schedule, Cost, and Management Issues at DOE’s Hanford Tank Waste Project (GAO/T-RCED-99-21, Oct. 8, 1998).
Department of Energy: Problems in DOE’s Foreign Visitor Program Persist (GAO/T-RCED-99-19, Oct. 6, 1998).
Nuclear Waste: Further Actions Needed to Increase the Use of Innovative Cleanup Technologies (GAO/RCED-98-249, Sept. 25, 1998).
Department of Energy: DOE Lacks an Effective Strategy for Addressing Recommendations From Past Laboratory Advisory Groups (GAO/T-RCED-98-274, Sept. 23, 1998).
Department of Energy: Uncertain Progress in Implementing National Laboratory Reforms (GAO/RCED-98-197, Sept. 10, 1998).
Department of Energy: Lessons Learned Incorporated Into Performance- Based Incentive Contracts (GAO/RCED-98-223, July 29, 1998).
Information Technology: Department of Energy Does Not Effectively Manage Its Supercomputers (GAO/RCED-98-208, July 17, 1998).
Financial Management: Fostering the Effective Implementation of Legislative Goals (GAO/T-AIMD-98-215, June 18, 1998).
DOE Management: Functional Support Costs at DOE Facilities (GAO/RCED-98-193R, June 12, 1998).
DOE Fiscal Year 1999 Budget Request for Energy Efficiency and Renewable Energy and Financial Management Issues (GAO/RCED-98-186R, June 10, 1998).
Department of Energy: Alternative Financing and Contracting Strategies for Cleanup Projects (GAO/RCED-98-169, May 29, 1998).
Results Act: Observations on DOE’s Annual Performance Plan for Fiscal Year 1999 (GAO/RCED-98-194R, May 28, 1998).
Department of Energy: Clear Strategy on External Regulation Needed for Worker and Nuclear Facility Safety (GAO/T-RCED-98-205, May 21, 1998).
Department of Energy: Clear Strategy on External Regulation Needed for Worker and Nuclear Facility Safety (GAO/RCED-98-163, May 21, 1998).
Nuclear Waste: Management Problems at the Department of Energy’s Hanford Spent Fuel Storage Project (GAO/T-RCED-98-119, May 12, 1998).
Department of Energy: DOE Contractor Employee Training (GAO/RCED-98-155R, May 8, 1998).
Department of Energy: Problems and Progress in Managing Plutonium (GAO/RCED-98-68, Apr. 17, 1998).
Results Act: DOE Can Improve Linkages Among Plans and Between Resources and Performance (GAO/RCED-98-94, Apr. 14, 1998).
Nuclear Weapons: Design Reviews of DOE’s Tritium Extraction Facility (GAO/RCED-98-75, Mar. 31, 1998).
Nuclear Waste: Understanding of Waste Migration at Hanford Is Inadequate for Key Decisions (GAO/RCED-98-80, Mar. 13, 1998).
Best Practices: Elements Critical to Successfully Reducing Unneeded RDT&E Infrastructure (GAO/NSIAD/RCED-98-23, Jan. 8, 1998).
Department of Energy: Subcontracting Practices (GAO/RCED-98-30R, Nov. 24, 1997).
Department of Energy: Information on the Tritium Leak and Contractor Dismissal at the Brookhaven National Laboratory (GAO/RCED-98-26, Nov. 4, 1997).
Department of Energy: Clearer Missions and Better Management Are Needed at the National Laboratories (GAO/T-RCED-98-25, Oct. 9, 1997).
Department of Energy: DOE Needs to Improve Controls Over Foreign Visitors to Weapons Laboratories (GAO/RCED-97-229, Sept. 25, 1997).
Results Act: Observations on the Department of Energy’s August 15, 1997, Draft Strategic Plan (GAO/RCED-97-248R, Sept. 2, 1997).
Results Act: Observations on Federal Science Agencies (GAO/T-RCED-97-220, July 30, 1997).
Nuclear Waste: Department of Energy’s Pit 9 Cleanup Project Is Experiencing Problems (GAO/T-RCED-97-221, July 28, 1997).
Nuclear Waste: Department of Energy’s Project to Clean Up Pit 9 at Idaho Falls Is Experiencing Problems (GAO/RCED-97-180, July 28, 1997).
Results Act: Comments on Selected Aspects of the Draft Strategic Plans of the Departments of Energy and the Interior (GAO/T-RCED-97-213, July 17, 1997).
Results Act: Observations on the Department of Energy’s Draft Strategic Plan (GAO/RCED-97-199R, July 11, 1997).
Department of Energy: Status of DOE’s Efforts to Improve Training (GAO/RCED-97-178R, June 27, 1997).
High-Risk Program: Information on Selected High-Risk Areas (GAO/HR-97-30, May 1997).
Department of Energy: Opportunity for Enhanced Oversight of Major System Acquisitions (GAO/RCED-97-146R, Apr. 30, 1997).
Department of Energy: Information on the Distribution of Funds for Counterintelligence Programs and the Resulting Expansion of These Programs (GAO/RCED-97-128R, Apr. 25, 1997).
Department of Energy: Funding and Workforce Reduced, but Spending Remains Stable (GAO/RCED-97-96, Apr. 24, 1997).
Department of Energy: Plutonium Needs, Costs, and Management Programs (GAO/RCED-97-98, Apr. 17, 1997).
Department of Energy: Improving Management of Major System Acquisitions (GAO/T-RCED-97-92, Mar. 6, 1997).
Department of Energy: Management and Oversight of Cleanup Activities at Fernald (GAO/RCED-97-63, Mar. 14, 1997).
High-Risk Series: Department of Energy Contract Management (GAO/HR-97-13, Feb. 1997).
Nuclear Waste: DOE’s Estimates of Potential Savings From Privatizing Cleanup Projects (GAO/RCED-97-49R, Jan. 31, 1997).
Nuclear Waste: Impediments to Completing the Yucca Mountain Repository Project (GAO/RCED-97-30, Jan. 17, 1997).
Department of Energy: Contract Reform Is Progressing, but Full Implementation Will Take Years (GAO/RCED-97-18, Dec. 10, 1996).
Department of Energy: Opportunity to Improve Management of Major System Acquisitions (GAO/RCED-97-17, Nov. 26, 1996).
DOE Security: Information on Foreign Visitors to the Weapons Laboratories (GAO/T-RCED-96-260, Sept. 26, 1996).
Department of Energy: Observations on the Future of the Department (GAO/T-RCED-96-224, Sept. 4, 1996).
Hanford Waste Privatization (GAO/RCED-96-213R, Aug. 2, 1996).
Nuclear Weapons: Improvements Needed to DOE’s Nuclear Weapons Stockpile Surveillance Program (GAO/RCED-96-216, July 31, 1996).
Information Management: Energy Lacks Data to Support Its Information System Streamlining Effort (GAO/AIMD-96-70, July 23, 1996).
Energy Management: Technology Development Program Taking Action to Address Problems (GAO/RCED-96-184, July 9, 1996).
DOE’s Cleanup Cost Savings (GAO/RCED-96-163R, July 1, 1996).
DOE’s Laboratory Facilities (GAO/RCED-96-183R, June 26, 1996).
Energy Research: Opportunities Exist to Recover Federal Investment in Technology Development Projects (GAO/RCED-96-141, June 26, 1996).
Department of Energy: Progress Made Under Its Strategic Alignment and Downsizing Initiative (GAO/T-RCED-96-197, June 12, 1996).
Federal Facilities: Consistent Relative Risk Evaluations Needed for Prioritizing Cleanups (GAO/RCED-96-150, June 7, 1996).
Managing DOE: The Department’s Efforts to Control Litigation Costs (GAO/T-RCED-96-170, May 14, 1996).
Energy Downsizing: While DOE Is Achieving Budget Cuts, It Is Too Soon to Gauge Effects (GAO/RCED-96-154, May 13, 1996).
Success Stories Response (GAO/OCG-96-3R, May 13, 1996).
DOE Cleanup: Status and Future Costs of Uranium Mill Tailings Program (GAO/T-RCED-96-167, May 1, 1996).
DOE’s Success Stories Report (GAO/RCED-96-120R, Apr. 15, 1996).
Environmental Protection: Issues Facing the Energy and Defense Environmental Management Programs (GAO/T-RCED/NSIAD-96-127, Mar. 21, 1996).
Nuclear Weapons: Status of DOE’s Nuclear Stockpile Surveillance Program (GAO/T-RCED-96-100, Mar. 13, 1996).
Federal R&D Laboratories (GAO/RCED/NSIAD-96-78R, Feb. 29, 1996).
Nuclear Nonproliferation: Concerns With the U.S. International Nuclear Materials Tracking System (GAO/T-RCED/AIMD-96-91, Feb. 28, 1996).
Uranium Mill Tailings: Status and Future Costs of Cleanup (GAO/T-RCED-96-85, Feb. 28, 1996).
Energy’s Financial Resources and Workforce (GAO/RCED-96-69R, Feb. 28, 1996).
Nuclear Waste: Management and Technical Problems Continue to Delay Characterizing Hanford’s Tank Waste (GAO/RCED-96-56, Jan. 26, 1996).
Uranium Mill Tailings: Cleanup Continues, but Future Costs Are Uncertain (GAO/RCED-96-37, Dec. 15, 1995).
Department of Energy: A Framework for Restructuring DOE and Its Missions (GAO/RCED-95-197, Aug. 21, 1995).

	Other Reports

Report to Congress on the Plan for Organizing the National Nuclear Security Administration (Department of Energy, National Nuclear Security Administration, May 3, 2001).
Special Report: Performance Measures at the Department of Energy (Department of Energy, Office of Inspector General, DOE/IG-0504, May 2001).
Prepared Testimony of John A. Gordon, Under Secretary of Energy and Administrator for Nuclear Security, National Nuclear Security Administration, U.S. Department of Energy, Before the Senate Appropriations Committee, Energy & Water Subcommittee (Apr. 26, 2001).
Statement of John A. Gordon, Under Secretary of Energy and Administrator for Nuclear Security, National Nuclear Security Administration, U.S. Department of Energy, Before the Special Oversight Panel on Department of Energy Reorganization, Committee on Armed Services, U.S. House of Representatives (Apr. 4, 2001).
Audit Report: Bechtel Jacobs Company LLC’s Management and Integration Contract at Oak Ridge (Department of Energy, Office of Inspector General, DOE/IG-0498, Mar. 21, 2001).
Science and Technology Issues Facing the 107th Congress: First Session (Congressional Research Service-RL30869, Mar. 1, 2001).
Department of Energy: Performance and Accountability Report Fiscal Year 2000 (Department of Energy/CR-0071, Feb. 16, 2001).
Federal Managers’ Financial Integrity Act (Department of Energy, Memorandum for the Secretary of Energy from Gregory H. Friedman, Inspector General, CR-L-01-06, Feb. 8, 2001).
FY 2000 Report to Congress of the Panel to Assess the Reliability, Safety, and Security of the United States Nuclear Stockpile (Feb. 1, 2001).
Eleventh Annual Report to Congress (Defense Nuclear Facilities Safety Board, Feb. 2001).
H.R. 376—To Abolish the Department of Energy (107th Congress, Jan. 31, 2001).
Interim Letter Report for the Improved Project Management in the Department of Energy (The National Academies, Jan. 17, 2001).
The Department of Energy’s Tritium Production Program (Congressional Research Service-RL30425, Jan. 12, 2001).
Nuclear Energy Policy (Congressional Research Service-IB88090, Jan. 12, 2001).
Civilian Nuclear Waste Disposal (Congressional Research Service- IB92059, Jan. 10, 2001).
The National Ignition Facility: Management, Technical, and Other Issues (Congressional Research Service-RL30540, Jan. 4, 2001).
Department of Energy Research and Development Budget for FY 2001: Description and Analysis (Congressional Research Service-RL30445, Jan. 3, 2001).
Annual Performance Plan for FY 2001 (Department of Energy/CR-0068-9).
China: Suspected Acquisition of U.S. Nuclear Weapon Secrets (Congressional Research Service-RL30143, Dec. 20, 2000).
DOE Science for the Future: A Discussion Paper (Academic Panel, Dec. 14, 2000).
Performance-Based Management at the Department of Energy (External Members of the Laboratory Operations Board, Dec. 7, 2000).
Contributions and Value of the Laboratory Operations Board (Department of Energy, Memorandum from Ernest Moniz and John McTague to Bill Richardson, Secretary of Energy, Dec. 7, 2000).
Special Report: Management Challenges at the Department of Energy (Department of Energy, Office of Inspector General, DOE/IG-0491, Nov. 28, 2000).
United States Department of Energy: Fact Book FY 2000 (Department of Energy, Office of Management and Administration, Office of Management and Operations Support, Nov. 2000).
Establishing the National Nuclear Security Administration: A Year of Obstacles and Opportunities (Special Oversight Panel on Department of Energy Reorganization, Committee on Armed Services, U.S. House of Representatives, Oct. 13, 2000).
DOE’s Civilian Information Technology Program (Congressional Research Service-RS20626, Oct. 5, 2000).
Field Restructuring (Department of Energy, Memorandum for Heads of Departmental Elements from T. J. Glauthier, Sept. 26, 2000).
Restructuring DOE and Its Laboratories: Issues in the 106th Congress (Congressional Research Service-IB10036, Sept. 13, 2000).
The Department of Energy’s Spallation Neutron Source Project: Description and Issues (Congressional Research Service-RL30385, Sept. 12, 2000).
Strategic Plan: Powering the 21st Century—Strength Through Science (Department of Energy/CR-0070, Sept. 2000).
Audit Report: Security Overtime at the Oak Ridge Operations Office (Department of Energy, Office of Inspector General, ER-B-00-02, June 21, 2000).
Roles and Responsibilities Guiding Principles (Department of Energy, Memorandum from T. J. Glauthier to Under Secretary, Energy, Science, and Environment and Acting Administrator for Nuclear Security, June 2, 2000).
Audit Report: Central Shops at Brookhaven National Laboratory (Department of Energy, Office of Inspector General, ER-B-00-01, May 11, 2000).
Audit Report: Performance Incentives at the Idaho National Engineering and Environmental Laboratory (Department of Energy, Office of Inspector General, WR-B-00-05, Apr. 3, 2000).
Statement of Dan W. Reicher, Assistant Secretary for Energy Efficiency and Renewable Energy, U.S. Department of Energy, Before the Subcommittee on Interior and Related Agencies, Committee on Appropriations, U.S. House of Representatives Oversight Hearing on Energy Conservation Financial Management Procurement (Mar. 30, 2000).
Charitable Giving Requirements in Department of Energy Contracts (Department of Energy, Memorandum From the Inspector General to the Deputy Secretary, HQ-L-00-01, Mar. 14, 2000).
A Review of Management in the Office of Energy Efficiency and Renewable Energy (National Academy of Public Administration, Mar. 2000).
Audit Report: The Department’s Management and Operating Contractor Make-or-Buy Program (Department of Energy, Office of Inspector General, DOE/IG-0460, Feb. 17, 2000).
Tenth Annual Report to Congress (Defense Nuclear Facilities Safety Board, Feb. 2000).
Strength Through Science—U.S. Department of Energy FY 2001 Budget Request to Congress—Budget Highlights (Department of Energy, Office of Chief Financial Officer, Feb. 2000)
Congress and the Fusion Energy Sciences Program: A Historical Analysis (Congressional Research Service-RL30417, Jan. 31, 2000).
Audit Report: Follow-up Audit of Program Administration by the Office of Science (Department of Energy, Office of Inspector General, DOE/IG-0457, Jan. 24, 2000).
Audit Report: The Management of Tank Waste Remediation at the Hanford Site (Department of Energy, Office of Inspector General, DOE/IG-0456, Jan. 21, 2000).
Audit Report: Outsourcing Opportunities at the Los Alamos National Laboratory (Department of Energy, Office of Inspector General, WR-B-00-03, Jan. 18, 2000).
Research and Development Budget of the Department of Energy for FY2000: Description and Analysis (Congressional Research Service- RL30054, Dec. 16, 1999).
Inspection Report: Inspection of Alleged Improprieties Regarding Issuance of a Contract (DOE/IG-INS-O-00-02, Dec. 16, 1999).
Contractor Make or Buy Plan Implementation (Department of Energy, Memorandum from Richard Hopf, Director, Office of Procurement and Assistance Management, to Heads of Contracting Activities, Dec. 6, 1999).
Stockpile Stewardship Program: 30-Day Review (Department of Energy, Nov. 23, 1999).
FY 1999 Report of the Panel to Assess the Reliability, Safety, and Security of the United States Nuclear Stockpile (Nov. 8, 1999).
Department of Energy: Programs and Reorganization Proposals (Congressional Research Service-RL30307, Sept. 17, 1999).
DOE Security: Protecting Nuclear Material and Information (Congressional Research Service-RS20243, July 23, 1999).
Glauthier Announces DOE Project Management Reforms (Department of Energy Press Release, June 25, 1999).
Technology Transfer to China: An Overview of the Cox Committee Investigation Regarding Satellites, Computers, and DOE Laboratory Management (Congressional Research Service-RL30231, June 11, 1999).
Science at its Best, Security at its Worst: A Report on Security Problems at the U.S. Department of Energy (A Special Investigative Panel, President’s Foreign Intelligence Advisory Board, June 1999).
Changes to the Departmental Management Structure (Department of Energy, Memorandum from the Secretary of Energy to Heads of Departmental Elements, Apr. 21, 1999).
Commission on Maintaining United States Nuclear Weapons Expertise: Report to the Congress and Secretary of Energy (Mar. 1, 1999).
Ninth Annual Report to Congress (Defense Nuclear Facilities Safety Board, Feb. 1999).
Audit Report: The U.S. Department of Energy’s Implementation of the Government Performance and Results Act (Department of Energy, Office of Inspector General, DOE/IG-0439, Feb. 4, 1999).
U.S. National Security and Military/Commercial Concerns with the People’s Republic of China (Select Committee, United States House of Representatives, Jan. 3, 1999).
Department of Energy: Accountability Report Fiscal Year 1999 (Department of Energy/CR-0069, 1999).
Improving Project Management in the Department of Energy (National Research Council, 1999).
U.S. Department of Energy Strategic Alignment Initiative, Fiscal Year 1998 Status Report (Department of Energy, 1999).
Audit Report: The U.S. Department of Energy’s Efforts to Increase the Financial Responsibility of Its Major For-Profit Operating Contractors (Department of Energy, Office of Inspector General, DOE/IG-0432, Nov. 20, 1998).
Audit Report: Project Hanford Management Contract Costs and Performance (Department of Energy, Office of Inspector General, DOE/IG-0430, Nov. 5, 1998).
Audit Report: The U.S. Department of Energy’s Prime Contractor Fees on Subcontractor Costs (Department of Energy, Office of Inspector General, DOE/IG-0427, Sept. 11, 1998).
Unlocking Our Future: Toward a New National Science Policy (A Report to Congress by the House Committee on Science, Sept. 24, 1998).
Audit Report: The Cost Reduction Incentive Program at the Savannah River Site (Department of Energy, Office of Inspector General, ER-B-98-08, May 29, 1998).
Inspection Report: The Fiscal Year 1996 Performance Based Incentive Program at the Savannah River Operations Office (Department of Energy, Office of Inspector General, May 1998).
Assessing the Need for Independent Project Reviews in the DOE (National Research Council, 1998).
Audit of Support Services Subcontracts at Argonne National Laboratory (Department of Energy, Office of the Inspector General, DOE/IG-0416, Dec. 23, 1997).
Departmental Reporting Relationships (Department of Energy, Memorandum from J. M. Wilcynski, Manager, Idaho Operations Office, to the Deputy Secretary and Under Secretary, Nov. 26, 1997).
Audit Report: Audit of the Contractor Incentive Program at the Nevada Operations Office (Department of Energy, Office of Inspector General, DOE/IG-0412, Oct. 20, 1997).
Restructuring DOE and Its Laboratories: Issues in the 105th Congress (Congressional Research Service-IB97012, Oct. 15, 1997).
External Members of the Laboratory Operations Board Analysis of Headquarters and Field Structure Issues (Secretary of Energy Advisory Board, Oct. 2, 1997).
DOE Laboratory Restructuring Legislation in the 104th Congress (Congressional Research Service-97-558SPR, May 13, 1997).
The Organization and Management of the Nuclear Weapons Program: 120-Day Study (Institute for Defense Analysis, Feb. 27, 1997).
Seventh Annual Report to Congress (Defense Nuclear Facilities Safety Board, Feb. 1997).
Department of Energy Strategic Alignment Initiative Status Report— Fiscal Year 1996 (DOE, Dec. 1996).
How to Close Down the Department of Energy (The Heritage Foundation, Nov. 9, 1995).
Department of Energy Abolition? Implications for the Nuclear Weapons Program (Congressional Research Service-95-1020F, Sept. 29, 1995).
Strategic Alignment: Tracking Our Progress (Department of Energy, Sept. 5, 1995).
Energy R&D: Shaping Our Nation’s Future in a Competitive World (Final Report of the Task Force on Strategic Energy Research and Development, June 1995).
Alternative Futures for the DOE National Laboratories (Task Force on Alternative Futures for the National Laboratories (Secretary of Energy Advisory Board, Feb. 1995).

Appendix III: Comments From the Department of Energy

The following are GAO's comments on the Department of Energy's letter dated November 30, 2001.

	GAO Comments

1.  Our response is included in the body of the report. 2.
In our report, we acknowledge and support DOE’s efforts to implement performance-based contracting practices and to competitively award more of its contracts.  As suggested, we have revised our report to note that the department has not been required to compete contracts to manage its Federally Funded Research and Development Centers. 3.  As we state in our report, our concern is that some of DOE's largest contracts, notably those with the University of California to manage several national laboratories, have never been opened to competitive bidding.  According to DOE, the decisions related to the most recent contract extension with this university were based on "national security considerations " and were not "contract management decisions …" The benefits of competing contracts are widely accepted and espoused by DOE in its own policies.  Recent interest shown by another university in competing for the Sandia National Laboratory contract when it expires in 2003 suggests that there may be other capable competitors, and that national security considerations do not inhibit DOE from attracting new performers. 4.  We agree that DOE sponsors many "unique" projects that contain significant research and development that can impact cost and schedule assumptions, and we have incorporated this comment in our report.  Nevertheless, we concur with DOE that this circumstance should not be used as "an excuse for the poor performance in project management" that was cited in our report. 5.  We do not concur with DOE that the department’s strategic planning process has worked effectively to organize and integrate its diverse missions.  As we said in our report, DOE told us that its strategic plan is a composite of plans that guides the program activities of the department's four "business lines," each of which establishes its own objectives and management systems.  Acknowledging the unfocused nature of the department, the Secretary is just now taking steps to define an overarching departmental objective for all programs and to expand NNSA’s new Planning, Programming, Budgeting and Evaluation system department-wide.  He is also creating a new office under the Chief Financial Officer that "will analyze and evaluate plans, programs and budgets in relation to the department's objectives…" The department said that it expects this office will serve as the "linchpin" for making improvements in strategic planning in the future. 6.  We reported in 1998 that DOE's Strategic Laboratory Missions plan, which was published in 1996, was essentially a descriptive summary of current laboratory activities; it did not direct change.  Nor did the plan tie DOE's or the laboratories' missions to the annual budget process. As we previously reported, when we asked laboratory officials about strategic planning, most discussed their own planning capabilities, and some laboratories provided us with their own self-generated strategic planning documents.  None of the officials at the multiprogram laboratories we visited at the time mentioned DOE's Strategic Laboratory Missions plan as an essential document for their own strategic planning. 7.  We noted in our report that DOE is attempting to clarify roles and responsibilities.  We also noted that DOE's 1999 reorganization was similar to steps the department had taken previously without success. While we have not assessed the effectiveness of the new Field Management Council, we noted in our report that the establishment of the NNSA appears to have created, at least temporarily, additional confusion regarding roles, responsibilities, and reporting relationships within the department. 8.  We noted in our report that the department has been taking steps to address its workforce problems since the early 1990s, and it continues to do so today.  As we said, we are concerned by the lack of succession planning and progress by DOE in addressing known human capital deficiencies.  We have revised our report, however, to reflect that DOE published, in September 2001, its "Five-Year Workforce Restructuring Plan."  According to DOE, the plan responds to an OMB requirement of all federal agencies and presents a "corporate roadmap" for reducing manager and organizational layers, increasing spans of control, and redeploying staff.  The plan describes a variety of ongoing and planned actions.  Regarding DOE's discussion of the many underlying factors affecting its staffing, we agree that building a quality workforce is very challenging.  As DOE notes, these challenges are made more difficult by the constant changes in mission focus that characterize DOE's history.

Appendix IV: GAO Contacts and Staff Acknowledgments

	GAO Contacts

	Staff Acknowledgments

In addition to those named above, Tom Laetz, Dan Feehan, William Lanouette, Tom Kingham, Linda Chu, James Charlifue, and Cynthia Norris made key contributions to this report.

Output:
Why GAO Did This Study

The Department of Energy (DOE) manages the nation's nuclear weapons production complex, cleans up the environmental legacy from the production of nuclear weapons, and conducts research and development into both energy and basic science. DOE launched several reforms in the 1990s to realign its organizational structure, reduce its workforce, strengthen contracting procedures by competitive awards practices, streamline oversight of activities, and delegate some responsibilities to the private sector.

What GAO Found

Despite these reforms, GAO found that management weaknesses persist because DOE's reforms were piecemeal solutions whose effect has been muted by three impediments to fundamental improvement: the department's diverse missions, dysfunctional organizational structure, and weak control of accountability. Management weaknesses and performance problems will likely continue unless DOE addresses these impediments in a comprehensive fashion.