LOCKHEED SHUTTLE PROCESSING CONTRACT-AWARD FEE HISTORY

Percent of Period From : ! ; :A Rating adjective Rating maximum Award fee mre award lee earned earned

¶First ............................................... Oct. 1, 1983 ............ $6,618,880 Excellent ............... 90.0 80 $5,295,104 Second........................................... Apr. 1, 1984 ............ 1,299,404 Good ..................... 78.5 32 415,809 Third .............................................. Oct. 1, 1984 ............ 1,308,554 (;ood ..................... 76.0 24 314,053 Fourth............................................ Apr. 1, 1985 ............ 1,308,554 Excellent ............... 91.0 84 1,099,185 Fifth............................................... Oct. 1, 1985 ............ 1,308,554 Very good ............. 89.0 76 994,501 Sixth .............................................. Apr.1,1986 ............ 1,296,664 ( 1 ) ....................... (I) (l) (')¹¹ g8 mid., July 24, 1986, p. 11. To be determined.

¶The rating scale runs from unacceptable to marginal, good, very good, excellent and superior. Two of the five ratings to date have been at the lower end of the scale.

¶At the time of the Challenger accident, Shuttle processing had suffered from inadequate spare parts for well over a year, and the problem was getting worse. The inventory of spare parts had run close to projections until the second quarter of fiscal year 1985. At that time, inventory requirements for spares began to increase faster than deliveries. A year later, the inventory should have been complete, but only 65 percent had been d e l i ~ e r e d . ' ~

¶The number of cannibalized parts was increasing at an alarming rate. Forty-five out of almost 300 required parts were cannibalized for Challenger before Mission 51-L.74 Eighty-five parts were cannibalized on 61-C, the mission preceding 51-L.75 In fact, the number of cannibalized parts on each of these 1986 missions far exceeded the number of cannibalized parts on any previous mission. In 14 missions flown in 1984-1985, the average number of cannibalized parts was 14; in the 1986 mission, the number had increased nearly five-fold to 65.

¶The cause of the spare parts crisis was budgetary decisionmaking by NASA management. In October, 1985, the logistics funding requirements for the Orbiter program, as determined by Level I11 management a t Johnson, were $285.3 million, but that funding was reduced by $83.3 million, necessitating major deferral of purchases of ~ p a r e s . ~By

6 the spring of 1986, the Shuttle logistics program 73 Rogers Commission Report, Volume 11, p. 1-16. 71 bid. 75 Cmte Hgs Transcript, July 16, 1986, p. 42. Rogers mmiasion Report, V o l u m e I, p. 173.

117

¶was about one year behind; and under the proposed flight schedule, no Orbiters would have been available as spare parts bins.

¶NASA is well aware of the spare parts problem. In fact, during the Committee's hearings, Admiral Truly testified:

I can assure you that during our downtime we're going to take a hard look at it and make sure that the flight rates that we build up to after this accident are support- able by the logistics system that we have in place.77

¶The Committee received mixed reactions on whether development contractors need to be more involved in the SPC. Proponents of this approach argue that the current separation of responsibilities between the design organizations and the processing organization has created additional interfaces which make coordination, communication, and responsiveness more complex. Further, the processing contractor may not possess the necessary technical background to recognize either system degradation resulting from multiple missions or the criticality of the hardware being tested and proces~ed.'~ The Rogers Commission report stated that the likelihood of improper Shuttle processing would probably be decreased if Rockwell, as overall development contractor, and Martin Marietta, who has a consulting role on the pre-launch processing of the External Tank, were subcontractors to Lockheed, as the other Shuttle development contractors are.T9

¶At Committee hearings, contractors reacted predictably to proposed changes in the SPC. Development contractors, such as Rockwell and Martin Marietta, told the Committee that their organizations should be vested with beginning-to-end responsibility-design, development, manufacturing, operation, and refurbishment of their respective Shuttle element.so Lockheed, on the other hand, argued that there is already a very close relationship between itself and the development contractors. For example, there is one development engineer for every four Lockheed engineers. Development contractors participate in all meetings and are required to authorize and approve anything that is anomalous to the regular documented procedure.8

¶Ultimately, SPC performance will determine the proper balance of development contractors in the processing contract. NASA, in close consultation with the Congress, will need to make an impar- tial and ongoing assessment of comparative safety and performance under a consolidated versus unconsolidated SPC. Preliminary figures from NASA seem to indicate that Shuttle processing incidents have actually declined during the more recent consolidated-contract phase.82 Further, it is likely that the fundamental problem to date with the SPC-overtime-would be exacerbated by the additional contractor coordination that would be required by greater inclusion of development contractors.

¶77 Cmte Hgs, Transcript, July 11, 1986, p. 39.

¶78 Rogers Commission Report, Volume 11, p. K-32.

¶'9 Rogers Commission Report, Volume I, p. 195.

Cmte Hgs, Transc+pt, July 15, 1986, p. 62. Cmte Hgs,Transcript, July 16, 1986, pp. 13-14.⁸⁸ Ibid. Ibid. ' 0 bid. I 1bid. Morton Thiokol, "Program Plan, Protection of Space Shuttle SRM Primary Seals," TWR- 14359,May 4, 1984. 2 NASA, documents on the SPC contract, supplied to the Committee in July, 1986; Cmte Hgs, Transcript, July 16,1986, p. 67, and Attachment C. The responsibility for high overtime rates in the SPC must be shared by both NASA and the contractor. Mr. E.D.Sargent, President of Lockheed Space Operations Company, testified:

118

One of the problems that bothers us and drives us to overtime is either unplanned work or another form of unplanned work which is a hold or abort on the pad where we have critical skills that are required to perform functions.

¶There is no doubt that late mission changes initiated by NASA are in large part responsible for Lockheed exceeding the five percent overtime target in the SPC contract.

¶But in fact overtime levels had grown from an initial SPC rate of 5.3 percent in April, 1984, to 13.9 percent in January, 1986-levels far in excess of what could be attributed solely to late mission changes. The peak monthly overtime level of 15.2 percent occurred in November, 1985. Although NASA managers a t Kennedy attribute the November rate to the Thanksgiving holiday, the overall trend in overtime is undeniable-for each of the six months prior to the launch of STS 51-L, overtime exceeded 10 percente8*

¶More important that the average overtime rates was the overtime for certain employees with critical skills. Records show that there was a frequent pattern at Kennedy of combining weeks of consecutive workdays with multiple strings of 11- or 12-hour days. For example, one Lockheed mechanical technician team leader worked 60, 96.5, 94, and 80.8 hours per week in succession during the four weeks ending January 31, 1986.85While shiftwork is commonplace in many industrial settings, few can equal a Shuttle launch's potential for inducing pressure to work beyond reasonable overtime limits.

¶Research has shown that when overtime becomes excessive, worker efficiency decreases and the potential for human error rises. Noteworthy in this regard is Lockheed's review of 264 incidents that caused property damage in 1984 and 1985. More than 50 percent of these incidents were attributable to human error, including procedural deviations, miscommunications and safety violations.86 On one occasion a potentially catastrophic error occurred just minutes before a scrubbed launch of Shuttle flight 61-C on January 6, 1986, when 18,000 pounds of liquid oxygen were inadvertently drained from the Shuttle's External Tank. The investigation which followed cited operator fatigue as one of the major factors contributing to this incident. The operators had been on duty at the console for eleven hours during the third day of working 12-hour night shifts. If the launch had not been held 31 seconds before lift off, the mission might not have achieved orbit.87

¶The adequacy of and adherence to Operations and Maintenance Instructions (OMI's) have been raised as areas of concern leading to quality and safety problems. Review of various SPC mishap reports and of the procedures leading to the launch of 51-L and earlier

119

¶Shuttle flights highlight both the need for review and update of inadequate OMI's and the need for improved contractor performance in implementing adequate O M I ' S . ~ ~

¶NASA's own review of flight 51-L showed several examples of improperly implemented procedures. The most serious error occurred when a console operator improperly closed the liquid hydrogen disconnect valve to the External Tank liquid hydrogen manifold. Although the valve appeared to function during 51-L, improper valve operation could have doomed 51-L just as surely as the failed rocket booster. As important as the failure to follow the OM1 was the fact that the valve closure problem was never documented. Without proper documentation a full assessment of the problem was not made prior to launch of 51-L.89 This lack of documentation is reminiscent of what occurred during "de-stacking" of Solid Rocket Motor segments from STS-9. Although destacking revealed water in the joints, this incident was never documented-an oversight which ultimately may have prevented an appreciation of the dangers of ice formation in booster joints during a cold-weather launch.s0 b. Pressures on Shuttle Operations

¶Issue

¶Was NASA under pressure to fly more flights? How did this pressure originate? Will it recur?

¶Findings

The Congress and the Executive Branch jointly developed the policy that the Space Shuttle should, in a reliable fashion and at an internationally competitive cost, provide for most of the Free World's space launch needs. By and large, both Branches failed to appreciate the impact that this policy was having on the operational safety of the system.
NASA was under internal and external pressure to build its Shuttle flight rate to 24 per year, primarily to reduce costs per flight, but also to demonstrate and achieve routine access to space. NASA has never achieved its planned flight rate.

¶Recommendations

NASA must not attempt to achieve a flight rate beyond that which (i) can be supported by the budget and staff resources available; and (ii) is consistent with the technical maturity of the Shuttle and the flexibility desired and needed in scheduling payloads. Management should ensure efficient use of resources but should not impose a flight rate on the system.
Once operation of the Space Shuttle resumes, the Committee should maintain a close and continuous oversight of Shuttle flight rate, planning, and operations. The Committee should ensure both that flight rate flows logically from the resources provided and that flight safety is not compromised beyond acceptable limits.

¶Kennedy Space Center Mishap Reports, No. 85-0070, April 5, 1985, and No. 86-0024, Dec. 13, 1985.

120

¶Discwwion

¶Flight Rate. The goal of the Shuttle program has been to become the Nation's primary space transportation system launching virtually all US. payloads and many foreign payloads, all at a reasonable price. Thus, there has been an explicit promise to deliver launch services.

¶Being a very complicated vehicle, the Shuttle demands a large trained workforce which must be retained between launches. In addition, there are the costs of maintaining large and complex launch facilities. Therefore, there is a large fixed cost in the Shuttle program of approximately $1.2 billion per year. By comparison the ad- ditive or marginal cost for a single fight is around $60 million (depending on how the accounting is done). Therefore, it is clear that (within limits) the cost-per-flight can be reduced by flying more flights, that is, by spreading the large fixed cost over more flights. However, it is also clear that the total cost-that is, the total amount of money that has to be appropriated-will increase as the number of flights increases because fixed costs are fixed and marginal costs must be added for each additional flight.

¶Therefore, to focus on cost-per-flight can be misleading. A lower cost-per-flight, achieved by flying more often, would allow a lower price to be charged to users, but does not lower the cost of the program. Because NASA had committed to lower the price to customers of Shuttle flights, there was a pressure to do this by increasing the flight rate. Nevertheless, NASA never achieved its planned flight rate.

¶For example, in 1976 NASA predicted 49 flights in fiscal 1984 and 58 in 1985..As late as August 15, 1983,⁴⁵⁴⁵ Ibid,. Volume 11, p. K-23. days before the start of fiscal year 1984, NASA planned 9 flights for fiscal 1984 and 12 for 1985. NASA actually flew four Shuttle flights in fiscal 1984 and 8 in 1985.91Of course management worked hard to reduce this gap between plans and performance.

¶The emphasis on reducing costs per flight and delivering launch services has caused a very basic and pervasive pressure to increase the flight rate in the Shuttle program. This is well documented in Chapter VIII of the Rogers Conimission report.92

¶Presumably, the Challenger accident has changed this situation. Recommendation VIII of the Rogers Commission states in part that "NASA must establish a flight rate that is consistent with its resource^."^ NASA's response to this recommendation hints that this may not be the case. NASA speaks of determining "the maximum achievable safe flight rate."g4 Such a flight rate would again leave no "margin in the system to accommodate unforeseen hardware problems" as the Commission found was the case before the accident.95 The NASA response makes it clear that the flight rate ~~~~~

¶9 1 Hearings before the House Committee on Science and Technology, FY 1978 NASA Authori- +ion, %p,Fmber 14, 1976, Volume I, Part 1, 394; NASA, "Space Shuttle Payload Flight Assignments. August 15, 1983. NASA "Space Jhuttle Payload Flight Assignments," November, 1%.⁹⁹ Rogers Commission Report, loc. cit. 1 Rogers Commission Report, Volume I, pp. 164-77.⁹⁹ Rogers Commission Report, loc. cit. 3 Wid., p. 201.⁹⁹ Rogers Commission Report, loc. cit. 4 NASA Response to Rogers Commission, Jul 14, 1986, pp. 30-31.⁹⁹ Rogers Commission Report, loc. cit. 5 Rogers Commission Report, Volume I, p. 17f.

121

¶will be determine based on studies and that "program enhancements . . . required to achieve the flight rate" will be implemented [emphasis added].96 This is reinforced in the NASA response to Recommendation IX where NASA says "NASA has initiated an assessment of spare parts requirements to adequately support the flight rate planning. " [emphasis added] g 7 Thus, it seems that once again a planned flight rate could become a controlling factor.

¶A finding of the Pre-Launch Activities Team is that during the preparation of 51-L for launch "Manpower limitations due to high workload created scheduling difficulties and contributed to operational problems."gs This is perhaps one of the clearest examples of the inappropriate logic at work in the system before the accident, because "manpower limitations" are not due to "high workload" in the system. Manpower and other resources are limited before the workload is planned. Problems are created when the workload assigned is inappropriate to the manpower available.

¶In a March 24, 1986, memorandum on "Strategy for Safely Returning the Space Shuttle to Flight Status," Admiral Truly reveals a better attitude toward flight rate in speaking of a "realistic and . . . achievable launch rate that will be safely sustainable." Admiral Truly also states that "the ultimate safe sustainable flight rate and the build up to that rate will be developed utilizing a 'bottoms up' approach in which all required work for the standard flow . . . is identified and that work is optimized in relation to the available work force."99

¶NASA prepared several reports for the Rogers Commission, and the Mission Planning and Operations Team (MPOT) Report indicates a good awareness of the general problem of over-ambitious flight rate planning. For example, that report states that compared to the need to devote resources to making the transition to an operational system the "increasing flight rate had the highest priority." The MPOT report continues, "In other words, it appears that the flight rate was not tied to the ability of the system to support it: but rather the system was reacting to the established flight rate. A major conclusion of the MPOT report is that "The NSTS Program should develop a bottoms-up strategy for expanding flight rate."loo In other words, flight rate cannot be imposed from above, but must be determined by available resources.

¶The disturbing fact is the trend in the NASA statements. The earlier statements (i.e., the Truly memo and the MPOT report) indicated an awareness of the danger of trying to achieve an imposed flight rate. However, as mentioned above, the most recent statement, the NASA response to the Commission, once again speaks of achieving the planned flight rate.

¶The Rogers Commission has documented the fact that before the Challenger accident the Shuttle system was approaching a state of saturation in which no more flights could be accommodated. If the accident had not occurred flight rate saturation may have eventually been reached due to bottlenecks in crew training on the mis- s 6 NASA Response to Rogers Commission, July 14, 1986, p. 31. 9'bid., p. 33. 9*Rogers Commission Report, Volume 11, p. 1-14, 99NASA Response to Rogers commission, July 14, 1986, p. 40.¹⁰⁰¹⁰⁰ Ibid., pp. 583-84. Rogers Commission Report, Volume 11, p. 5-31,

¶64-420 0 - 86 - 5 sion simulators O or because of inadequate spare parts for the Orbiter. l o 2

122

¶Availability of training time on simulators and availability of spare parts can both be improved by the application of more resources. Nevertheless, if the achievement of a planned flight rate is the overriding concern, removal of one bottleneck may only reveal another one. Eventually, pressures will be brought to bear on safety. The pressure on NASA to increase Shuttle flight rate has been complicated by the need to maintain program flexibility (which means to accommodate changes in the payloads on the manifest) and by the "developmental" nature of the Shuttle system. Manifest changes and the developmental nature of the system create problems in the planning of Shuttle missions.

¶In addition, it is interesting to note that until the training, spares, and mission planning problems are resolved, achievable flight rate may not depend on whether or not Challenger is replaced.

¶c. Impact of Pressures on Shuttle Operations

¶Issue

¶Did operating pressures adversely affect the safety of the Shuttle program?

¶Findings

The pressure on NASA to achieve planned flight rates was so pervasive that it undoubtedly adversely affected attitudes regarding safety.
The pressure to achieve planned flight rates was compressing mission preparation as earlier missions were delayed due to unforeseen problems. Had the accident not occurred there would soon have been a collision between planned launch dates and mission preparation needs which could not have been met by overtime, cannibalization, or other undesirable practices. Operating pressures were causing an increase in unsafe practices.
The schedule of payloads planned to fly on the Shuttle (the manifest) was frequently changed. Each change rippled through the NASA Shuttle organization and through the manifest and, especially if made shortly before launch, would increase the demands on personnel and resources in order to achieve the planned flight rate.
The Space Shuttle has not yet reached a level of maturity which could be called operational as that term is used in either the airline industry or the military. Each Shuttle flight is fundamentally unique, and requires unique preparations. Therefore, small changes in a mission can cause significant perturbations of mission planning and crew training.

¶Recommendations

The new Associate Administrator for Safety, Reliability and Quality Assurance must assure that any pressures to increase the

¶101 Ibid., Volume I, p. 170. 102 Ibid., p. 174.

123

¶Shuttle flight rate do not adversely influence mission preparation. The Associate Administrator must have the authority not only to stop a particular flight, e.g., at a Flight Readiness Review, but to stop the whole mission planning process if necessary.

Where appropriate, NASA should take steps to make the mission planning process standard and routine to reduce the time and resources needed to plan a mission. Before requesting more resources for the existing mission planning process (manpower, facilities, equipment) NASA should identify ways to improve the process. Discussion

¶There is no doubt that operating pressures created a n atmosphere which allowed the accident on 51-L to happen. Without operating pressures the program might have been stopped months before the accident to redesign or at least understand the SRB joint. Without operating pressure the flight could have been stopped the night of January 27. This is documented in the Rogers Commission report in Chapters V and VI.lo3 Specific manifesta- tions of launch pressure and the resultant atmosphere in the agency are described in detail in Section VIII of this report.

¶Nevertheless, it has become clear that the Shuttle launch system was not functioning well and was becoming increasingly unsafe as flight rate was increased. This is documented in Chapter VIII of the Rogers Commission report. l o 4

¶Mission Planning.-Mission planning refers to the process of defining and preparing each Space Shuttle mission. It is important to understand the mission planning process in order to understand why pressure to achieve a given flight rate could have adverse impacts. The process is lengthy, complex, and tightly interrelated. That is, many steps must be done in sequence, and many different flights have to use limited resources and facilities.

¶Mission planning begins at NASA headquarters with the customer services manager in the Office of Space Flight. Both financial and policy agreements between NASA and the customer are nego- tiated and signed. Technical documentation begins at this time although the level of mission-specific work is low.

¶After flight assignments are made by NASA Headquarters and the mission is defined, a process of continual review begins. Payloads are assigned to a particular flight 33 months prior to launch. At this time a Payload Integration Plan (PIP) is developed which includes a preliminary analysis of the mission.

¶Payload safety is the responsibility of the payload developer. He must be throughly familiar with NASA safety requirements and must certify that his payload meets them. NASA audits the certification process but performs no visual inspection of the payload for conformance to safety standards.

¶Once the cargo of a particular mission has been defined, or "baselined", the significant engineering work of mission processing actually begins. NASA refers to this as the "production process". The product of the process is the launch of a particular mission,¹⁰³¹⁰³ Ibid., pp. 59g-97. Ibid., pp. 82-151. lo4 Ibid., pp. 164-77.

124

¶but there are many other intermediate products such as flight and training software, crew activity plans, and handbooks and check- lists for the crew to take on the flight.

¶The launch production process template is displayed schematically on Figure VI-2. The template begins 15 months before the scheduled launch date (L-13, at which time a Flight Definition and Requirements Directive (FDRD) is issued. This marks one of seven defined "freeze points" of the 15 month mission-specific pre-launch activity. A freeze point simply means that a particular activity is norminally defined so that no time changes can occur without a formal process to authorize and document the change. In theory, non-mandatory changes are not made after a freeze point. As noted below, significant changes do indeed occur after the various freeze points in the schedule.

GENERIC TEMPLATE (CIR 7.7 MO)

¶1s 14 13 12 11 10 9 8 7 6 5 4 3 2 1 s

¶I I I I I I I I I I I I I I I

¶I I I I

: -L I I I — 1

I I

¶I I I I . . I

¶I I

¶I

¶I I

I CREW nwuw uyc I

¶1 I

¶I

¶I I

¶I

¶< . M~DoQ)yyulWN

¶I I

¶I I I

¶I 1 I

I I I I I Oll*T€@ SYSTEMSh W V % S I I I I I I I I I I I I I I L-10.9 I I I I

0 L.3 b.5 1-7.7

¶VI-2