7.6 FINDINGS AND RECOMMENDATIONS

¶The evidence that supports the organizational causes also led the Board to conclude that NASAʼs current organization, which combines in the Shuttle Program all authority and responsibility for schedule, cost, manifest, safety, technical requirements, and waivers to technical requirements, is not an effective check and balance to achieve safety and mission assurance. Further, NASAʼs Office of Safety and Mission Assurance does not have the independence and authority that the Board and many outside reviews believe is necessary. Consequently, the Space Shuttle Program does not consistently demonstrate the characteristics of organizations that effectively manage high risk. Therefore, the Board offers the following Findings and Recommendations:

¶Findings:

¶F7.1-1 Throughout its history, NASA has consistently

struggled to achieve viable safety programs and adjust them to the constraints and vagaries of changing budgets. Yet, according to multiple high level independent reviews, NASAʼs safety system has fallen short of the mark.

193

¶F7.4-1 The Associate Administrator for Safety and Mis-

sion Assurance is not responsible for safety and mission assurance execution, as intended by the Rogers Commission, but is responsible for Safety and Mission Assurance policy, advice, coordination, and budgets. This view is consistent with NASAʼs recent philosophy of management at a strategic level at NASA Headquarters but contrary to the Rogersʼ Commission recommendation.

¶F7.4-2 Safety and Mission Assurance organizations sup-

porting the Shuttle Program are largely dependent upon the Program for funding, which hampers their status as independent advisors.

¶F7.4-3 Over the last two decades, little to no progress has

been made toward attaining integrated, independent, and detailed analyses of risk to the Space Shuttle system.

¶F7.4-4 System safety engineering and management is

separated from mainstream engineering, is not vigorous enough to have an impact on system design, and is hidden in the other safety disciplines at NASA Headquarters.

¶F7.4-5 Risk information and data from hazard analyses

are not communicated effectively to the risk assessment and mission assurance processes. The Board could not find adequate application of a process, database, or metric analysis tool that took an integrated, systemic view of the entire Space Shuttle system.

¶F7.4-6 The Space Shuttle Systems Integration Office

handles all Shuttle systems except the Orbiter. Therefore, it is not a true integration office.

¶F7.4-7 When the Integration Office convenes the Inte-

gration Control Board, the Orbiter Office usually does not send a representative, and its staff makes verbal inputs only when requested.

¶F7.4-8 The Integration office did not have continuous

responsibility to integrate responses to bipod foam shedding from various offices. Sometimes the Orbiter Office had responsibility, sometimes the External Tank Office at Marshall Space Flight Center had responsibility, and sometime the bipod shedding did not result in any designation of an In-Flight Anomaly. Integration did not occur.

¶F7.4-9 NASA information databases such as The Prob-

lem Reporting and Corrective Action and the R7.5-2 Web Program Compliance Assurance and Status System are marginally effective decision tools.

¶F7.4-10 Senior Safety, Reliability & Quality Assurance

and element managers do not use the Lessons R7.5-3 Learned Information System when making decisions. NASA subsequently does not have a constructive program to use past lessons to edu- cate engineers, managers, astronauts, or safety personnel.

¶F7.4-11 The Space Shuttle Program has a wealth of data

tucked away in multiple databases without a convenient way to integrate and use the data for management, engineering, or safety decisions.

¶F7.4-12 The dependence of Safety, Reliability & Quality

communicate potential problems throughout the organization.

¶7.4-13 There are conflicting roles, responsibilities, and

guidance in the Space Shuttle safety programs. The Safety & Mission Assurance Pre-Launch Assessment Review process is not recognized by the Space Shuttle Program as a requirement that must be followed (NSTS 22778). Failure to consistently apply the Pre-Launch Assessment Review as a requirements document creates confusion about roles and responsibilities in the NASA safety organization.

¶ecommendations:

¶7.5-1 Establish an independent Technical Engineer-

ing Authority that is responsible for technical requirements and all waivers to them, and will build a disciplined, systematic approach to identifying, analyzing, and controlling hazards throughout the life cycle of the Shuttle System. The independent technical authority does the following as a minimum:

Develop and maintain technical standards for all Space Shuttle Program projects and elements

Be the sole waiver-granting authority for all technical standards

Conduct trend and risk analysis at the subsystem, system, and enterprise levels

Own the failure mode, effects analysis and hazard reporting systems

Conduct integrated hazard analysis

Decide what is and is not an anomalous event

Independently verify launch readiness

Approve the provisions of the recertification program called for in Recommendation R9.1-1

The Technical Engineering Authority should be funded directly from NASA Headquarters, and should have no connection to or responsibility for schedule or program cost.

¶7.5-2 NASA Headquarters Office of Safety and Mission

Assurance should have direct line authority over the entire Space Shuttle Program safety organization and should be independently resourced.

¶7.5-3 Reorganize the Space Shuttle Integration Office

to make it capable of integrating all elements of the Space Shuttle Program, including the Orbiter.

Assurance personnel on Shuttle Program support limits their ability to oversee operations and

194

¶The citations that contain a reference to "CAIB document" with CAB or CTF followed by seven to eleven digits, such as CAB001-0010, refer to a document in the Columbia Accident Investigation Board database maintained by the Department of Justice and archived at the National Archives.

Sylvia Kramer, "History of NASA Safety Office from 1958-1980ʼs," NASA History Division Record Collection, 1986, p. 1. CAIB document

¶CAB065-0358.²² before the 1986 Challenger accident NASA adopted an alternate Space For detailed discussions of the origins of the Space Shuttle, see Dennis R. Shuttle mission numbering scheme, this report uses the original STS flight Jenkins, Space Shuttle: The History of the National Space Transportation designations. System – The First 100 Missions (Cape Canaveral, FL: Specialty Press, 12 2001); T. A. Heppenheimer, The Space Shuttle Decision: NASAʼs Search President Reaganʼs quote is contained in President Ronald Reagan, for a Reusable Space Vehicle, NASA SP-4221 (Washington: Government "Remarks on the Completion of the Fourth Mission of the Space Shuttle Printing Office, 1999; also published by the Smithsonian Institution Press, Columbia," July 4, 1982, p. 870, in Public Papers of the Presidents of the 2002); and T. A. Heppenheimer, Development of the Space Shuttle, United States: Ronald Reagan (Washington: Government Printing Office, 1972-1981 (Washington: Smithsonian Institution Press, 2002). Much of 1982-1991). The emphasis noted is the Boardʼs. the discussion in this section is based on these studies. 13 "Pricing Options for the Space Shuttle," Congressional Budget Office National Commission on Space, Pioneering the Space Frontier: An Exciting Vision of Our Next Fifty Years in Space, Report of the National Commission on Space (Bantam Books, 1986), p. 2. Guidelines per NASA Policy Guideline 8621.Show full note

Ralph M. Miles Jr. "Introduction." In Ralph M. Miles Jr., editor, System Concepts: Lectures on Contemporary Approaches to Systems, p. 1-12 (New York: John F. Wiley & Sons, 1973).

¶"The Aerospace Safety Advisory Panel, " NASA History Office, July 1,

¶1987, p. 1.⁴⁴ See also comments by Robert F. Thompson, Columbia Accident Shuttle Development," June 8, 1986, p. 1-7. Investigation Board Public Hearing, April 23, 2003, in Appendix G. 16 The 1971 cost-per-flight estimate was $7.7 million; $140.5 million dollars "Report of the Advisory Committee on the Future of the U.S. Space Program," December 1990, p. 2. 212 Report Volume I August 2003 JSC Form 564 (March 24, 2003).Show full note

¶On Rodneyʼs appointment, see NASA Management Instruction 1103.39,

¶July 3, 1986, and NASA News July 8, 1986.⁵⁵ Heppenheimer, The Space Shuttle Decision, pp. 278-289, and Roger in 1985 when adjusted for inflation becomes $52.9 million in 1971 A. Pielke, Jr., "The Space Shuttle Program: ʻPerformance vs. Promise,ʼ" dollars or nearly seven times the 1971 estimate. "Pricing Options for the Center for Space and Geosciences Policy, University of Colorado, August Space Shuttle." 31, 1991; Logsdon, "The Space Shuttle Program: A Policy Failure?" pp. 17 See Diane Vaughan, The Challenger Launch Decision: Risky Technology, 1099-1105. Culture, and Deviance at NASA (Chicago: The University of Chicago U.S.C. App § §1 et seq. (1972). Harold W. Gehman to Sean OʼKeefe, February 25, 2003. 238 Report Volume I August 2003Show full note

NASA Facts, "Brief Overview, Office of Safety, Reliability, Maintainability and Quality Assurance," circa 1987.

¶"Space Program Safety: Funding for NASAʼs Safety Organizations

Should Be Centralized," General Accounting Office Report, NSIAD-90- 187, 1990.

¶"Aerospace Safety Advisory Panel Annual Report," 1996.⁸⁸ The only actual flight tests conducted of the Orbiter were a series of Printing Office, 1998) for an account of the aftermath of the accident. Approach and Landing Tests where Enterprise (OV-101) was dropped Much of the account in this section is drawn from this source. from its Boeing 747 Shuttle Carrier Aircraft while flying at 25,000 feet. 19 Logsdon, "Return to Flight," p. 348. These tests – with crews aboard – demonstrated the low-speed handling 20 Presidential Commission on the Space Shuttle Challenger Accident capabilities of the Orbiter and allowed an evaluation of the vehicleʼs (Washington: Government Printing Office, June 6, 1986). landing characteristics. See Jenkins, Space Shuttle, pp. 205-212 for more information. 26 Report Volume I August 2003Show full note

¶The quotes are from the Executive Summary of National Aeronautics

and Space Administration Space Shuttle Independent Assessment Team, "Report to Associate Administrator, Office of Space Flight," October- December 1999. CAIB document CTF017-0169.

Harry McDonald, "SIAT Space Shuttle Independent Assessment Team Report."

¶NASA Chief Engineer and NASA Integrated Action Team, "Enhancing

¶Mission Success – A Framework for the Future," December 21, 2000.¹¹¹¹ Thompson, "The Risk of Groundling Fatalities;" Code of Federal Regulations (CFR) 14 CFR Part 415, 415, and 417, "Licensing and Safety Requirements for Launch: Proposed Rule," Federal Register Vol. 67, No. 146, July 30, 2002, p. 49495. 224 Report Volume I August 2003

¶The information in this section is derived from a briefing titled, "Draft

Final Report of the Space Shuttle Competitive Source Task Force," July 12, 2002. Mr. Liam Sarsfield briefed this report to NASA Headquarters.

Dr. Karl Weick, University of Michigan; Dr. Karlene Roberts, University of California-Berkley; Dr. Howard McCurdy, American University; and Dr. Diane Vaughan, Boston College.

Dr. David Woods, Ohio State University; Dr. Nancy G. Leveson, Massachusetts Institute of Technology; Mr. James Wick, Intel Corporation; Ms. Deborah L. Grubbe, DuPont Corporation; Dr. M. Sam

Mannan, Texas A&M University; Douglas A. Wiegmann, University of Illinois at Urbana-Champaign; and Mr. Alan C. McMillan, President and Chief Executive Officer, National Safety Council.

Todd R. La Porte and Paula M. Consolini, "Working in Practice but Not in Theory," Journal of Public Administration Research and Theory,¹¹ program. NASA actually spent $9.9 billion in real year dollars to George Mueller, Associate Administrator for Manned Space Flight, take the Shuttle through design, development and initial testing. This NASA, "Honorary Fellowship Acceptance," address delivered to the sum, when converted to fixed year 1971 dollars using the aerospace British Interplanetary Society, University College, London, England, price deflator, equals $5.9 billion, or a 15 percent cost overrun on August 10, 1968, contained in John M. Logsdon, Ray A. Williamson, the original estimate for phase one. Compared to other complex Roger D. Launius, Russell J. Acker, Stephen J. Garber, and Jonathan L. development programs, this was not a large cost overrun." See Howard Friedman, editors, Exploring the Unknown: Selected Documents in the McCurdy, "The Cost of Space Flight," Space Policy 10 (4) p. 280. For History of the U.S. Civil Space Program Volume IV: Accessing Space, a program budget summary, see Jenkins, Space Shuttle, p. 256. NASA SP-4407 (Washington: Government Printing Office, 1999), pp. 11 202-205. STS stands for Space Transportation System. Although in the years just Report on the Commission on the Future of the United States Aerospace Industry, November 2002, p. 3-3. NASA Agency Contingency Action Plan for Space Flight Operations, January 2003, p. D-2.Show full note (1991)

¶pp. 19-47.

¶Scott Sagan, The Limits of Safety (Princeton: Princeton University Press,

¶1995). 16

¶Dr. Diane Vaughan, Boston College; Dr. David Woods, Ohio State

University; Dr. Howard E. McCurdy, American University; Dr. Karl E. Weick, University of Michigan; Dr. Karlene H. Roberts; Dr. M.

¶Elisabeth Paté-Cornell; Dr. Douglas A. Wiegmann, University of Illinois at Urbana-Champaign; Dr. Nancy G. Leveson, Massachusetts Institute of

Dupont Corporation; Dr. M. Sam Mannan, Texas A&M University; and Mr. Alan C. McMillan, President and Chief Executive Officer, National Safety Council.

Dr. David Woods of Ohio State University speaking to the Board on Hind- Sight Bias. April 28, 2003.

¶Sagan, The Limits of Safety, p.258. 19

¶LaPorte and Consolini, "Working In Practice." 20

Notes from "NASA/Navy Benchmarking Exchange (NNBE), Interim Report, Observations & Opportunities Concerning Navy Submarine Program Safety Assurance," Joint NASA and Naval Sea Systems Command NNBE Interim Report, December 20, 2002.

Theodore Rockwell, The Rickover Effect, How One Man Made a Difference. (Annapolis, Maryland: Naval Institute Press, 1992), p. 318.

¶Rockwell, Rickover, p. 320. 23

For more information, see Dr. Diane Vaughn, The Challenger Launch Decision, Risky Technology, Culture, and Deviance at NASA (Chicago: University of Chicago Press, 1996).

Presentation to the Board by Admiral Walter Cantrell, Aerospace Advisory Panel member, April 7, 2003.

Aerospaceʼs Launch Verification Process and its Contribution to Titan Risk Management, Briefing given to Board, May 21, 2003, Mr. Ken Holden, General Manager, Launch Verification Division.

Joe Tomei, "ELV Launch Risk Assessment Briefing," 3rd Government/ Industry Mission Assurance Forum, Aerospace Corporation, September 24, 2002.

NASA Policy Directive 8700.1A, "NASA Policy for Safety and Mission Success", Para 1.b, 5.b(1), 5.e(1), and 5.f(1).

¶Charles B. Perrow. Normal Accidents (New York: Basic Books, 1984). 30

A. Shenhar, "Project management style and the space shuttle program (part 2): A retrospective look," Project Management Journal, 23 (1), pp. 32-37.

Harry McDonald, "SIAT Space Shuttle Independent Assessment Team Report."

¶Ibid. 33

"Post Challenger Evaluation of Space Shuttle Risk Assessment and Management Report, National Academy Press 1988," section 5.1, pg. 40.

¶Harry McDonald, "SIAT Space Shuttle Independent Assessment Team

¶Report." 35

¶NSTS-22254 Rev B. Ibid.

GAO Report, "Survey of NASA Lessons Learned," GAO-01-1015R, September 5, 2001.

¶E. Tufte, Beautiful Evidence (Cheshire, CT: Graphics Press). [in press.] 39

¶Ibid., Edward R. Tufte, "The Cognitive Style of PowerPoint," (Cheshire,

¶CT: Graphics Press, May 2003).

¶Technology; Mr. James Wick, Intel Corporation; Ms. Deborah L. Grubbe, Ibid.

195

History As Cause: Columbia and Challenger

¶The Board began its investigation with two central questions about NASA decisions. Why did NASA continue to fly with known foam debris problems in the years preceding the Columbia launch, and why did NASA managers conclude that the foam debris strike 81.9 seconds into Columbiaʼs flight was not a threat to the safety of the mission, despite the concerns of their engineers?