7.4 Organizational Causes: A Broken Safety Culture

¶Perhaps the most perplexing question the Board faced during its seven-month investigation into the Columbia accident was "How could NASA have missed the signals the foam was sending?" Answering this question was a challenge. The investigation revealed that in most cases, the Human Space Flight Program is extremely aggressive in reducing threats to safety. But we also know – in hindsight – that detection of the dangers posed by foam was impeded by "blind spots" in NASAʼs safety culture.

¶From the beginning, the Board witnessed a consistent lack of concern about the debris strike on Columbia. NASA managers told the Board "there was no safety-of-flight issue" and "we couldnʼt have done anything about it anyway." The investigation uncovered a troubling pattern in which Shuttle Program management made erroneous assumptions about the robustness of a system based on prior success rather than on dependable engineering data and rigorous testing.

185

¶The Shuttle Programʼs complex structure erected barriers to effective communication and its safety culture no longer asks enough hard questions about risk. (Safety culture refers to an organizationʼs characteristics and attitudes – promoted by its leaders and internalized by its members – that serve to make safety the top priority.) In this context, the Board believes the mistakes that were made on STS-107 are not isolated failures, but are indicative of systemic flaws that existed prior to the accident. Had the Shuttle Program observed the principles discussed in the previous two sections, the threat that foam posed to the Orbiter, particularly after the STS-112 and STS-107 foam strikes, might have been more fully appreciated by Shuttle Program management.

¶In this section, the Board examines the NASAʼs safety policy, structure, and process, communication barriers, the risk assessment systems that govern decision-making and risk management, and the Shuttle Programʼs penchant for substituting analysis for testing.

¶NASAʼs Safety: Policy, Structure, and Process

¶Safety Policy

¶NASAʼs current philosophy for safety and mission assurance calls for centralized policy and oversight at Head-

¶NASA Administrator

¶Code M

¶Office of Space Flight AA

Deputy AA JSC Center Director

ISS/SSP

¶Verbal Input

ISS Program Space Shuttle JSC Organization SR & QA Director Manager Program Managers Manager

Shuttle Element Managers Endorse Space Shuttle Space Shuttle Organization S & MA Manager Managers

¶Funding via Integrated Task Agreements

United Space Alliance Vice President SQ & MA

¶quarters and decentralized execution of safety programs at the enterprise, program, and project levels. Headquarters dictates what must be done, not how it should be done. The operational premise that logically follows is that safety is the responsibility of program and project managers. Managers are subsequently given flexibility to organize safety efforts as they see fit, while NASA Headquarters is charged with maintaining oversight through independent surveillance and assessment.28 NASA policy dictates that safety programs should be placed high enough in the organization, and be vested with enough authority and seniority, to "maintain independence." Signals of potential danger, anomalies, and critical information should, in principle, surface in the hazard identification process and be tracked with risk assessments supported by engineering analyses. In reality, such a process demands a more independent status than NASA has ever been willing to give its safety organizations, despite the recommendations of numerous outside experts over nearly two decades, including the Rogers Commission (1986), General Accounting Office (1990), and the Shuttle Independent Assessment Team (2000).

¶Safety Organization Structure

¶Center safety organizations that support the Shuttle Program are tailored to the missions they perform. Johnson and

Issue: Same Individual,⁴⁴ JSC Form 564 (March 24, 2003). roles that cross Center, Program and

¶or Headquarters responsibilies

Result: Failure of checks and balances

¶(Safety Advisor)

Code Q Safety and Mission Assurance AA

¶Code Q MMT Letter

Space Shuttle SR & QA Manager

JSC SR & QA

¶Director

Independent Space Shuttle Division Chief Assessment Office

¶Responsibility

¶Policy/Advice

¶Figure 7.4-1. Independent safety checks and balance failure.

186

¶Marshall Safety and Mission Assurance organizations are organized similarly. In contrast, Kennedy has decentralized its Safety and Mission Assurance components and assigned them to the Shuttle Processing Directorate. This management change renders Kennedyʼs Safety and Mission Assurance structure even more dependent on the Shuttle Program, which reduces effective oversight.

¶At Johnson, safety programs are centralized under a Director who oversees five divisions and an Independent Assessment Office. Each division has clearly-defined roles and responsibilities, with the exception of the Space Shuttle Division Chief, whose job description does not reflect the full scope of authority and responsibility ostensibly vested in the position. Yet the Space Shuttle Division Chief is empowered to represent the Center, the Shuttle Program, and NASA Headquarters Safety and Mission Assurance at critical junctures in the safety process. The position therefore represents a critical node in NASAʼs Safety and Mission Assurance architecture that seems to the Board to be plagued by conflict of interest. It is a single point of failure without any checks or balances.

¶Johnson also has a Shuttle Program Safety and Mission Assurance Manager who oversees United Space Allianceʼs safety organization. The Shuttle Program further receives program safety support from the Centerʼs Safety, Reliability, and Quality Assurance Space Shuttle Division. Johnsonʼs Space Shuttle Division Chief has the additional role of Shuttle Program Safety, Reliability, and Quality Assurance Manager (see Figure 7.4-1). Over the years, this dual designation has resulted in a general acceptance of the fact that the Johnson Space Shuttle Division Chief performs duties on both the Centerʼs and Programʼs behalf. The detached nature of the support provided by the Space Shuttle Division Chief, and the wide band of the positionʼs responsibilities throughout multiple layers of NASAʼs hierarchy, confuses lines of authority, responsibility, and accountability in a manner that almost defies explanation.

¶A March 2001 NASA Office of Inspector General Audit Report on Space Shuttle Program Management Safety Observations made the same point:

¶The job descriptions and responsibilities of the Space

¶Shuttle Program Manager and Chief, Johnson Safety

¶Office Space Shuttle Division, are nearly identical with each official reporting to a different manager. This overlap in responsibilities conflicts with the SFOC [Space

¶Flight Operations Contract] and NSTS 07700, which requires the Chief, Johnson Safety Office Space Shuttle

¶Division, to provide matrixed personnel support to the

¶Space Shuttle Program Safety Manager in fulfilling requirements applicable to the safety, reliability, and quality assurance aspects of the Space Shuttle Program.

¶The fact that Headquarters, Center, and Program functions are rolled-up into one position is an example of how a carefully designed oversight process has been circumvented and made susceptible to conflicts of interest. This organizational construct is unnecessarily bureaucratic and defeats NASAʼs stated objective of providing an independent safety function. A similar argument can be made about the placement of quality assurance in the Shuttle Processing Divisions at Kennedy, which increases the risk that quality assurance personnel will become too "familiar" with programs they are charged to oversee, which hinders oversight and judgment.

¶The Board believes that although the Space Shuttle Program has effective safety practices at the "shop floor" level, its operational and systems safety program is flawed by its dependence on the Shuttle Program. Hindered by a cumbersome organizational structure, chronic understaffing, and poor management principles, the safety apparatus is not currently capable of fulfilling its mission. An independent safety structure would provide the Shuttle Program a more effective operational safety process. Crucial components of this structure include a comprehensive integration of safety across all the Shuttle programs and elements, and a more independent system of checks and balances.

¶Safety Process

¶In response to the Rogers Commission Report, NASA established what is now known as the Office of Safety and Mission Assurance at Headquarters to independently monitor safety and ensure communication and accountability agency-wide. The Office of Safety and Mission Assurance monitors unusual events like "out of family" anomalies and establishes agency-wide Safety and Mission Assurance policy. (An out-of-family event is an operation or performance outside the expected performance range for a given parameter or which has not previously been experienced.) The Office of Safety and Mission Assurance also screens the Shuttle Programʼs Flight Readiness Process and signs the Certificate of Flight Readiness. The Shuttle Program Manager, in turn, is responsible for overall Shuttle safety and is supported by a one-person safety staff.

¶The Shuttle Program has been permitted to organize its safety program as it sees fit, which has resulted in a lack of standardized structure throughout NASAʼs various Centers, enterprises, programs, and projects. The level of funding a program is granted impacts how much safety the Program can "buy" from a Centerʼs safety organization. In turn, Safety and Mission Assurance organizations struggle to antici- pate program requirements and guarantee adequate support for the many programs for which they are responsible.

¶It is the Boardʼs view, shared by previous assessments, that the current safety system structure leaves the Office of Safety and Mission Assurance ill-equipped to hold a strong and central role in integrating safety functions. NASA Headquarters has not effectively integrated safety efforts across its culturally and technically distinct Centers. In addition, the practice of "buying" safety services establishes a relationship in which programs sustain the very livelihoods of the safety experts hired to oversee them. These idiosyncra- sies of structure and funding preclude the safety organization from effectively providing independent safety analysis.

¶The commit-to-flight review process, as described in Chapters 2 and 6, consists of program reviews and readiness polls that are structured to allow NASAʼs senior leaders to assess mission readiness. In like fashion, safety organizations affiliated with various projects, programs, and Centers at NASA, conduct a Pre-launch Assessment Review of safety preparations and mission concerns. The Shuttle Program does not officially sanction the Pre-launch Assessment Review, which updates the Associate Administrator for Safety and Mission Assurance on safety concerns during the Flight Readiness Review/Certification of Flight Readiness process.

187

¶The Johnson Space Shuttle Safety, Reliability, and Quality Assurance Division Chief orchestrates this review on behalf of Headquarters. Note that this division chief also advises the Shuttle Program Manager of Safety. Because it lacks independent analytical rigor, the Pre-launch Assessment Review is only marginally effective. In this arrangement, the Johnson Shuttle Safety, Reliability, and Quality Assurance Division Chief is expected to render an independent assessment of his own activities. Therefore, the Board is concerned that the Pre-Launch Assessment Review is not an effective check and balance in the Flight Readiness Review.

¶Given that the entire Safety and Mission Assurance organization depends on the Shuttle Program for resources and simultaneously lacks the independent ability to conduct detailed analyses, cost and schedule pressures can easily and unintentionally influence safety deliberations. Structure and process places Shuttle safety programs in the unenviable position of having to choose between rubber-stamping engineering analyses, technical efforts, and Shuttle program decisions, or trying to carry the day during a committee meeting in which the other side almost always has more information and analytic capability.

¶NASA Barriers to Communication: Integration, Information Systems, and Databases

¶By their very nature, high-risk technologies are exceptionally difficult to manage. Complex and intricate, they consist of numerous interrelated parts. Standing alone, components may function adequately, and failure modes may be anticipated. Yet when components are integrated into a total system and work in concert, unanticipated interactions can occur that can lead to catastrophic outcomes.29 The risks inherent in these technical systems are heightened when they are produced and operated by complex organizations that can also break down in unanticipated ways. The Shuttle Program is such an organization. All of these factors make effective communication – between individuals and between programs – absolutely critical. However, the structure and complexity of the Shuttle Program hinders communication.

¶The Shuttle Program consists of government and contract personnel who cover an array of scientific and technical disciplines and are affiliated with various dispersed space, research, and test centers. NASA derives its organizational complexity from its origins as much as its widely varied missions. NASA Centers naturally evolved with different points of focus, a "divergence" that the Rogers Commission found evident in the propensity of Marshall personnel to resolve problems without including program managers outside their Center – especially managers at Johnson, to whom they officially reported (see Chapter 5).

¶Despite periodic attempts to emphasize safety, NASAʼs frequent reorganizations in the drive to become more efficient reduced the budget for safety, sending employees conflicting messages and creating conditions more conducive to the development of a conventional bureaucracy than to the maintenance of a safety-conscious research-and-development organization. Over time, a pattern of ineffective communication has resulted, leaving risks improperly defined, problems unreported, and concerns unexpressed.30 The question is, why?

¶The transition to the Space Flight Operations Contract – and the effects it initiated – provides part of the answer. In the Space Flight Operations Contract, NASA encountered a completely new set of structural constraints that hindered effective communication. New organizational and contractual requirements demanded an even more complex system of shared management reviews, reporting relationships, safety oversight and insight, and program information development, dissemination, and tracking.

¶The Shuttle Independent Assessment Teamʼs report documented these changes, noting that "the size and complexity of the Shuttle system and of the NASA/contractor relationships place extreme importance on understanding, communication, and information handling."31 Among other findings, the Shuttle Independent Assessment Team observed that:

The current Shuttle program culture is too insular
There is a potential for conflicts between contractual and programmatic goals
There are deficiencies in problem and waiver-tracking systems
The exchange of communication across the Shuttle program hierarchy is structurally limited, both upward and downward.32

¶The Board believes that deficiencies in communication, including those spelled out by the Shuttle Independent Assessment Team, were a foundation for the Columbia accident. These deficiencies are byproducts of a cumbersome, bureaucratic, and highly complex Shuttle Program structure and the absence of authority in two key program areas that are responsible for integrating information across all programs and elements in the Shuttle program.

¶Integration Structures

¶NASA did not adequately prepare for the consequences of adding organizational structure and process complexity in the transition to the Space Flight Operations Contract. The agencyʼs lack of a centralized clearinghouse for integration and safety further hindered safe operations. In the Boardʼs opinion, the Shuttle Integration and Shuttle Safety, Reliability, and Quality Assurance Offices do not fully integrate information on behalf of the Shuttle Program. This is due, in part, to an irregular division of responsibilities between the Integration Office and the Orbiter Vehicle Engineering Office and the absence of a truly independent safety organization.

¶Within the Shuttle Program, the Orbiter Office handles many key integration tasks, even though the Integration Office appears to be the more logical office to conduct them; the Orbiter Office does not actively participate in the Integration Control Board; and Orbiter Office managers are actually ranked above their Integration Office counterparts. These uncoordinated roles result in conflicting and erroneous information, and support the perception that the Orbiter Office is isolated from the Integration Office and has its own priorities.

188

¶The Shuttle Programʼs structure and process for Safety and Mission Assurance activities further confuse authority and responsibility by giving the Programʼs Safety and Mission Assurance Manager technical oversight of the safety aspects of the Space Flight Operations Contract, while simultaneously making the Johnson Space Shuttle Division Chief responsible for advising the Program on safety performance. As a result, no one office or person in Program management is responsible for developing an integrated risk assessment above the sub-system level that would provide a comprehensive picture of total program risks. The net effect is that many Shuttle Program safety, quality, and mission assurance roles are never clearly defined.

¶Safety Information Systems

¶Numerous reviews and independent assessments have noted that NASAʼs safety system does not effectively manage risk. In particular, these reviews have observed that the processes in which NASA tracks and attempts to mitigate the risks posed by components on its Critical Items List is flawed. The Post Challenger Evaluation of Space Shuttle Risk Assessment and Management Report (1988) concluded that:

¶The committee views NASA critical items list (CIL) waiver decision-making process as being subjective, with little in the way of formal and consistent criteria for approval or rejection of waivers. Waiver decisions appear to be driven almost exclusively by the design based Failure Mode Effects Analysis (FMEA)/CIL retention rationale, rather than being based on an integrated assessment of all inputs to risk management.

¶The retention rationales appear biased toward proving that the design is "safe," sometimes ignoring significant evidence to the contrary.

¶The report continues, "… the Committee has not found an independent, detailed analysis or assessment of the CIL retention rationale which considers all inputs to the risk assessment process."33 Ten years later, the Shuttle Independent Assessment Team reported "Risk Management process erosion created by the desire to reduce costs …" 34 The Shuttle Independent Assessment Team argued strongly that NASA Safety and Mission Assurance should be restored to its previous role of an independent oversight body, and Safety and Mission Assurance not be simply a "safety auditor."

¶The Board found similar problems with integrated hazard analyses of debris strikes on the Orbiter. In addition, the information systems supporting the Shuttle – intended to be tools for decision-making – are extremely cumbersome and difficult to use at any level.

¶The following addresses the hazard tracking tools and major databases in the Shuttle Program that promote risk management. • Hazard Analysis: A fundamental element of system safety is managing and controlling hazards. NASAʼs only guidance on hazard analysis is outlined in the

¶Methodology for Conduct of Space Shuttle Program

¶Hazard Analysis, which merely lists tools available.35

¶Therefore, it is not surprising that hazard analysis processes are applied inconsistently across systems, subsystems, assemblies, and components.

¶United Space Alliance, which is responsible for both

¶Orbiter integration and Shuttle Safety Reliability and

¶Quality Assurance, delegates hazard analysis to Boeing. However, as of 2001, the Shuttle Program no longer requires Boeing to conduct integrated hazard analyses. Instead, Boeing now performs hazard analysis only at the sub-system level. In other words, Boeing analyzes hazards to components and elements, but is not required to consider the Shuttle as a whole. Since the current Failure Mode Effects Analysis/Critical Item

¶List process is designed for bottom-up analysis at the component level, it cannot effectively support the kind of "top-down" hazard analysis that is needed to inform managers on risk trends and identify potentially harmful interactions between systems.

¶The Critical Item List (CIL) tracks 5,396 individual

¶Shuttle hazards, of which 4,222 are termed "Critical-

SPACE SHUTTLE SAFETY UPGRADE PROGRAM

¶NASA presented a Space Shuttle Safety Upgrade Initiative to Congress as part of its Fiscal Year 2001 budget in March 2000. This initiative sought to create a "Pro-active upgrade program to keep Shuttle flying safely and efficiently to 2012 and beyond to meet agency commitments and goals for human access to space."

¶The planned Shuttle safety upgrades included: Electric Auxiliary Power Unit, Improved Main Landing Gear Tire, Orbiter Cockpit/Avionics Upgrades, Space Shuttle Main Engine Advanced Health Management System, Block III Space Shuttle Main Engine, Solid Rocket Booster Thrust Vector Control/Auxiliary Power Unit Upgrades Plan, Redesigned Solid Rocket Motor – Propellant Grain Geometry Modification, and External Tank Upgrades – Friction Stir Weld. The plan called for the upgrades to be completed by 2008.

¶However, as discussed in Chapter 5, every proposed safety upgrade – with a few exceptions – was either not approved or was deferred.

¶The irony of the Space Shuttle Safety Upgrade Program was that the strategy placed emphasis on keeping the "Shuttle flying safely and efficiently to 2012 and beyond," yet the Space Flight Leadership Council accepted the upgrades only as long as they were financially feasible. Funding a safety upgrade in order to fly safely, and then canceling it for budgetary reasons, makes the concept of mission safety rather hollow.

189

ity 1/1R." Of those, 3,233 have waivers. CRIT 1/1R component failures are defined as those that will result in loss of the Orbiter and crew. Waivers are granted whenever a Critical Item List component cannot be redesigned or replaced. More than 36 percent of these waivers have not been reviewed in 10 years, a sign that NASA is not aggressively monitoring changes in system risk.

¶It is worth noting that the Shuttleʼs Thermal Protection

¶System is on the Critical Item List, and an existing hazard analysis and hazard report deals with debris strikes.

¶As discussed in Chapter 6, Hazard Report #37 is ineffectual as a decision aid, yet the Shuttle Program never challenged its validity at the pivotal STS-113 Flight

¶Readiness Review.

¶Although the Shuttle Program has undoubtedly learned a great deal about the technological limitations inherent in Shuttle operations, it is equally clear that risk

¶– as represented by the number of critical items list and waivers – has grown substantially without a vigorous effort to assess and reduce technical problems that increase risk. An information system bulging with over

¶5,000 critical items and 3,200 waivers is exceedingly difficult to manage.

Hazard Reports: Hazard reports, written either by the Space Shuttle Program or a contractor, document conditions that threaten the safe operation of the Shuttle. Managers use these reports to evaluate risk and justify flight.36 During mission preparations, contractors and Centers review all baseline hazard reports to ensure they are current and technically correct.

¶Board investigators found that a large number of hazard reports contained subjective and qualitative judgments, such as "believed" and "based on experience from previous flights this hazard is an ʻAccepted Risk.ʼ" A critical ingredient of a healthy safety program is the rigorous implementation of technical standards. These standards must include more than hazard analysis or low-level technical activities. Standards must integrate project engineering and management activities. Finally, a mechanism for feedback on the effectiveness of system safety engineering and management needs to be built into procedures to learn if safety engineering and management methods are weakening over time.

¶Dysfunctional Databases

¶In its investigation, the Board found that the information systems that support the Shuttle program are extremely cumbersome and difficult to use in decision-making at any level. For obvious reasons, these shortcomings imperil the Shuttle Programʼs ability to disseminate and share critical information among its many layers. This section explores the report databases that are crucial to effective risk management.

Problem Reporting and Corrective Action: The Problem Reporting and Corrective Action database

records any non-conformances (instances in which a requirement is not met). Formerly, different Centers and contractors used the Problem Reporting and Corrective Action database differently, which prevented comparisons across the database. NASA recently initiated an effort to integrate these databases to permit anyone in the agency to access information from different Centers. This system, Web Program Compliance Assurance and Status System (WEBPCASS), is supposed to provide easier access to consolidated information and facilitates higher-level searches.

However, NASA safety managers have complained that the system is too time-consuming and cumbersome. Only employees trained on the database seem capable of using WEBPCASS effectively. One particularly frustrating aspect of which the Board is acutely aware is the databaseʼs waiver section. It is a critical information source, but only the most expert users can employ it effectively. The database is also incomplete. For instance, in the case of foam strikes on the Thermal Protection System, only strikes that were declared "In-Fight Anomalies" are added to the Problem Reporting and Corrective Action database, which masks the full extent of the foam debris trends.

Lessons Learned Information System: The Lessons Learned Information System database is a much simpler system to use, and it can assist with hazard identification and risk assessment. However, personnel familiar with the Lessons Learned Information System indicate that design engineers and mission assurance personnel use it only on an ad hoc basis, thereby limiting its utility. The Board is not the first to note such deficiencies. Numerous reports, including most recently a General Accounting Office 2001 report, highlighted fundamental weaknesses in the collection and sharing of lessons learned by program and project managers.37

¶Conclusions

¶Throughout the course of this investigation, the Board found that the Shuttle Programʼs complexity demands highly effective communication. Yet integrated hazard reports and risk analyses are rarely communicated effectively, nor are the many databases used by Shuttle Program engineers and managers capable of translating operational experiences into effective risk management practices. Although the Space Shuttle system has conducted a relatively small number of missions, there is more than enough data to generate performance trends. As it is currently structured, the Shuttle Program does not use data-driven safety methodologies to their fullest advantage.