Topics covered: Ground Operations ‑ Launching the Shuttle
Instructor: Guest Lecturer ‑ Bob Sieck
Subtitles are provided through the generous support of Heather Wood.
We're quite lucky today to have Bob Sieck talk to us about launch operations.
Bob joined NASA early in his career after a spell with the Air Force.
This was in the early '60s.
And he worked on prelaunch checkout and servicing of Gemini and Apollo.
Then went into the Shuttle Program.
He worked on ground operations for the Approach and Landing Tests.
We will hear more about that testing when Gordon Fullerton comes here at the beginning of December.
And then he worked as Launch Director for a lot of the early Shuttle missions, including my first flight.
And then went to be the Shuttle Operations Director at Kennedy Space Center.
But then after the Challenger accident he came back and worked on another, I don't know, 40 or so launches.
I think my last launch was in '96.
You had already graduated, right?
But Bob was the Launch Director of the first four of my flights.
He is the head of the whole team of really thousands of people who are responsible for getting the Shuttle ready to fly and for certifying that, in fact, it is ready to fly.
It is a very complicated, critical and from the safety point of view absolutely one of the most important things we do, is to figure out when this incredibly complex vehicle is really ready to go.
Bob is going to talk to us both about some of the technical aspects of the launch operations.
But also I hope, from a systems engineering point of view, we will have a chance towards the end to talk about some of the planning that was going on while the Shuttle was being designed and some of the compromises which had to be made, some of which we've already addressed, between the efficiency of turning around and maintaining a vehicle like the Shuttle vis-Ã -vis the actually initial upfront cost of it.
With that as an introduction, Bob, I will turn it over to you.
And good morning.
As Jeff indicated, I got into the business of launching people and payloads into space a long time ago.
I was privileged to be part of all of the human space flight programs, so I did a lot of that.
What I didn't do, I saw most of.
And I have an opinion on all of it, so please ask questions.
If I don't have the answer, I am sure to have an opinion.
I am going to spend a little bit of time, we talk about launch operations, but that's the final three to four days of a campaign that begins three to four months before you actually start the countdown clocks and put the astronauts in the vehicle and go fly.
So I am going to spend some time on the background of preparing the Shuttle, the ground systems, the flight systems prior to the launch count process particularly to talk about the engineering involvement and the responsibilities in those processes.
But, in order to talk about launching, I have to talk some about the operations of putting the final pieces together and doing the tests and inspections prior to launch.
And then I want to get into the real meat of it which is the human factors, the role of the engineer, the role of the managers in the process of testing, inspecting, checking the flight systems, the ground systems and certifying that it is OK to put the crew in and go fly.
And that's where we're going to spend most of our time, but I've got to go through some background first.
I don't know how many of you have been to the Kennedy Space Center.
Anybody been down there?
OK, so you've seen it.
It is a city where we've launched everything since the beginning programs, Mercury, Gemini, Apollo.
The most prominent fixture is this Vehicle Assembly Building which was used for the Apollo Program to put together the 300 foot tall Saturn-Apollo rocket.
Shuttle is about half that size but we use this building to do the assembly.
And I will talk a little bit about that later.
But it is a city.
We have our own power systems.
We have facilities to check out the orbiter to process the solid rocket motors.
We have administrative homes for the people.
And, of course, we have the highly visible launch pads that are out on the ocean.
And we also have a runway.
When the weather cooperates and we can bring the shuttles back to Florida, we can take care of the landing and recovery there.
We'll talk a little bit about the process.
When an orbiter returns from its previous mission, it goes through essentially a disassembly, inspection and then build it back up and test it to make sure that it is still within the certification base that the engineers set up for it that you've already heard from previous to this lecture.
You would like to think that if the Orbiter flies and the rest of the element is a successful mission you can just do it like an airplane, take down the down cargo, whatever, get it configured for the next payload that is going to go up, clean the windshield, so to speak, and the crew cabin and go fly it again.
But the rules don't allow you to do that.
The rules that the other engineers who have talked to you have set up are such that you have to assume that in some way, even though during the mission everything seemed to work OK, that that hardware was compromised through the process of reentering, landing, refurbishing those systems that have to be refurbished.
Something may have been invalidated.
So you have to test all that stuff that worked just fine in the previous mission, plus all of the systems that were not used in the previous mission, particularly the backup system, the things that you would have to rely on in the event that the primary system doesn't cooperate during the next mission.
And, because of the way the Orbiter was built with its miles of wiring and thousands of components that are active components, you have to disassemble a lot of that stuff in order to meet the requirements that the program is levied on.
After the Orbiter lands and you get it safe and the crew is out and you take out the payload, we spend a minimum of two to three months, roughly a quarter of a million labor hours of touch labor on the Orbiter to do this disassembly, test, inspection, refurbishment process before we move it from the hanger, take it to that Vehicle Assembly Building, integrate it with the tank and the solid rocket boosters and take it on out to the launch pad to go fly again.
And that's where the lion's share of the work is.
The work on the tank and the payloads and the solid rocket boosters is all peripheral to the processing of the Orbiter.
[AUDIENCE QUESTION] Do you find a lot of failures when you go through?
We find very few.
And you could do the tradeoff of, well, is this effort worth it?
But, on the other hand, the few that you find, you're glad that you found them because if not you may not have discovered them until you got out to the launch pad and serviced your hazardous systems.
Or, worse yet, you find out when you get in the mission.
And then the impact of that could be not only a safety impact but could be a higher consequence than spending the month to two months in an orbiter processing facility doing that work.
But is there a learning curve?
I mean after each mission do you know which system you have to look at specifically?
We look at it and say this system is performing just fine so we don't have to do this intense invasive inspection and test every flight.
Maybe we can just do it once a calendar year or every two to three missions, whatever the designer thinks is the criteria for checking on the health of this system.
And we've done that over the years.
And after we've had a couple of years of missions where everything is working just fine, we start relaxing these requirements.
And then something like Challenger happens or something like Columbia happens and the pendulum of conservatism swings the other direction.
And we do more tests and more inspections to verify the hardware because people want to be able to say, and we want to be able to say, well, we've done everything possible to make sure that that's a 100% functional, operating, safe orbiter.
And so getting this process down to where it meets the original advertising brochure which was we're going to fly these things every two to three weeks, you start in that direction.
And then events occur either during the mission or you have the really bad incidents like Challenger and Columbia.
And the conservatism swings the other way and you end up doing more work.
And, in addition to that, engineers, because we are what we are, I was one, we have all this hardware available, we like to modify it.
We change it.
We upgrade it because there is new technology available in computers or in composite materials.
So, when the orbiter gets to this processing facility, regardless of how well the mission flew, the designers, with their good intensions, say we want this orbiter to last another five years or ten years so we want you to do this upgrade to the avionics system or we want to change out all of the actuators that hold the payload in the payload bay and this and that.
So you modify.
And that adds more time.
It makes more complexity to the process which invites more tests and inspections.
You're constantly in a mode where unfortunately, if the mission flies perfectly, by the time you plan your campaign through the processing facility -- You plan your campaign with schedules to last two months maybe, roughly a third of the work you do is work that you hadn't planned when you initially brought that orbiter into the orbiter processing facility.
It's nonstandard work.
It is stuff that you uncover in the process of doing your test or inspection or its new requirements or new changes that come from the designers that have to be implemented.
And, from an efficiency standpoint, if you're in the manufacturing or production business, that is not efficient.
If a third of your work that you laid out for the next two months, at the end of that two month period, a third of it was stuff that you had not planned on doing which means you don't have the parts ready, you don't have the engineering, you don't have the instructions.
Do you build in time to cover that now?
Contingency time, yes.
Let me go back to that.
We set up our team and operations to run five days a week, two shifts a day leaving the weekends free to catch up for this nonstandard stuff.
We essentially end up running a six day work week if we're in a standard template of flying three missions a year, which we're not now because we're still recovering from the Columbia accident.
We end up working six days a week and roughly every three or four days we will work a continuous around the clock 24 hour operation.
We try to maintain that original schedule but we only make it roughly 50% of the time due to this, again, nonstandard work, the extra stuff that works its way in.
And that's why we have a big team.
Our team is made up of the United Space Alliance as a contractor.
They do the hands-on work.
They have the technicians.
They have the engineers that staff the consoles.
And they are roughly a 6,000 population at Kennedy Space Center.
They take care of the orbiter, the tank, the boosters.
They are responsible for maintaining all of our equipment that hooks into the shuttle that is required, the process of servicing it.
And they have ties to the contractors, the original equipment manufacturers that built the hardware.
Technically, they can gain access to all the original manufacturing records for the parts that are in the shuttle.
NASA's responsibility -- NASA owns the requirements for the Shuttle.
We delegated all the operational work to the contractors, but it's a national resource.
Taxpayers paid for it.
The government has a responsibility.
They cannot divest themselves from that.
NASA owns the requirements for saying what you test, what you inspect and what the specifications are for acceptable performance of that.
That is controlled by NASA.
The management of the contractor at Kennedy is done by NASA engineers in an organization that has roughly 500 people out of the 2000 population at Kennedy Space Center of civil service people.
500 are directly associated with the Shuttle.
The rest do payload work.
The Space Station, of course, is a big work item now at the Kennedy Space Center.
And they are responsible for the requirements.
If somebody wants to change the requirement, a NASA person has to approve that change.
And we do insight and everything the contractor does.
We have at each one of the milestones, when we leave the processing facility, when we leave the Vehicle Assembly Building and when we get ready to launch, there is a review conducted by NASA to review the work that was done by the contractor during the preceding weeks or months.
And NASA approves moving the vehicle onto the next stage of its assembly prior to launch.
And the launch director specifically is responsible not only for those last few days of the campaign, so to speak, leading up to launch, but also insuring the government contractor team has all the tools available to them to be successful.
Is the work being scheduled in an orderly fashion?
Is the ever-tightening budget constraining their ability to get their work done?
Is there too much scheduled pressure to meet a milestone?
That is the launch director's job.
The launch director doesn't just come in the last three days of the campaign with a tie and a suit and orchestrate the countdown.
They have a responsibility from the beginning to the end of the campaign.
I have already addressed this, but the NASA engineers manage the requirements, the changes to the requirements.
They are the ones in addition to the contractor, this is a check and balance, that look at all the data, the engineering results of the tests and the inspections.
And they approve that they met the requirements.
Also, they participate in critical operations.
A critical operation is a test on a specific system.
Servicing of the vehicle for launch count or any of the non-standard work that comes up during the process.
They audit what the contractor does.
And, of course, because we're the customer and they are the contractor, we have a lot of criteria to grade them on for their award fee performance.
The operations people manage the scheduling.
As I indicated, we like to work five days a week, two shifts a day.
We really end up working six days a week and sometimes around the clock.
The scheduling activity is dynamic.
And NASA approves all the scheduling.
We have to approve all the overtime.
We have the authority to say stop or go not only in launch count but anywhere in the campaign of processing the hardware, as well as the big picture schedule, you know, how many times a year you're going to fly and which orbiter is assigned to which payload in those missions.
That is still a NASA responsibility.
Orbiter processing facility, like I mentioned before, it is like a hanger for an airplane.
A big part of the work effort used to be the tile.
I think you already had a presentation on tile.
There are 25,000 to 30,000 of these little bricks glued to the vehicle.
We classically damage about a hundred of them on each mission due to ice and things coming off the tank that have to be replaced, but we also pull another hundred or so off just to inspect the integrity of the glue that is holding the tile to the structure of the
vehicle. Because a lot of those little bricks have been glued on there for 25 years and have flown 25 to 30 missions.
Although we have offline test programs that try to duplicate the environment that the tile sees either from a calendar standpoint or the thermal cycles of the mission, you would like to go to the real hardware and see what is happening.
We do that invasive type of work of actually destroying tile, pulling it off the vehicle just to make sure that the aging process is not causing any degradation of the system.
We pull the engines out and do the maintenance on the engines offline because the aft fuselage of the orbiter is a terrible place to work.
Any of the hardware we can get out of there and do it on the bench, it is much better to do that than to try to do it in the vehicle with people crawling all over wire bundles and structure and that sort of thing.
They are concerned about the collateral damage to do the other work in that compartment of the vehicle where the engines are.
Of course, you've got to take the accommodations from the previous mission out of the payload bay and put in the new stuff for the next mission.
I already mentioned modifications.
Unfortunately, we change more stuff than we probably should but in order to keep up with the aging of the vehicle and the new technology that goes with the territory of flying a system that was designed over 30 years ago.
And we prepared for the next milestone which was the vehicle assembly building operations.
This is where we put the big pieces together.
The solid rocket boosters are stacked.
There are four segments in each booster.
That is a very critical process but only takes less than a month to do.
The external tank comes in essentially ready to fly from Mississippi.
Except, as you know now, because of the Columbia accident, we are still looking at the ramifications of these lines and other fittings on the tank.
The acreage foam on the tank, just to digress a little bit, is sprayed on.
And that stuff is tough.
Early in the program we had problems with some of that coming off, but we solved that years ago.
What we haven't solved is where you have to do a manual application of foam over a fitting that is there because that is where the crane is attached to it or because there is a line there that has to be installed at the Kennedy Space Center.
You cannot do it back at the factory.
Putting that thermal protection system foam on there is a labor-intensive process.
It is a critical technique process.
And, obviously, there are still flaws in the process that the engineers haven't sorted out yet.
And that is why that stuff keeps coming off the tank in those places.
But the solid rocket motors are attached to this mobile launch platform with four bolts the diameter of my wrist.
The structure engineers really figured this out good.
There are four on each one of those that are attached to the launch platform.
The tank is attached to the boosters with four attach points, and those are bolts, again, that are the diameter of my wrist.
And then the orbiter is hung on the tank in three places with the same sized bolts.
You've got six million pounds of weight that is attached to the mobile launch platform that is held up by these four bolts, well, eight total on the solid rocket motors.
And they work out the dynamics of when the engines start because the vehicles sways like this because the center of gravity is not right over where those eight bolts are attached.
And that system works fine.
It always amazed me that you could hang that much weight on those few bolts that are only the diameter of this, but it works.
The orbiter is brought over after its campaign, it's hung on the tank and we test all those connections between the solid rocket booster, the platform, the tank and the orbiter.
And that only takes about a week to do that.
And, if that looks good, we have a review of all of that work.
And if the review says we didn't leave anything undone, that we shouldn't take out the launch pad then we go out to the launch pad.
[AUDIENCE QUESTION] Sure.
Say a little bit about this difference from just bringing your car into have a 50,000 mile checkup, that is the clean room aspects, the following of procedures, of having a checker.
Well, a 50,000 mile checkup on a car is usually an external inspection.
You change the oil and the filter and you check all the belts and the radiator hoses and you change all the fluids.
Well, you do that with the orbiter also.
But, in the case of the orbiter, you want to know whether you are this far away from having to do a valve job or the rings in the engine have now got so many thousand miles on them.
We have the equivalent.
You take the heads off the engine and you give it a valve job.
And you take the pistons out of the block and you check for scoring in the block.
You take the transmission out and you pull the gears out and make sure that the synchronizers are still OK.
And you go through the rear-end.
And you take all the avionics out.
Or, you take your carry-on test equipment in and you hook it up to these connectors on the airflow meter and the computer in your vehicle and you run all those diagnostics to make sure all that stuff, even though it worked fine after you turned that ignition key off the last time, that all the redundancies and all the capability is still within that hardware and all the margins are still there.
And then you button it back up and you make sure you button it back up properly, all of which is pre-established quite rigid flowcharts, checklists.
A checker checking the checker.
Oh, yes, and you have inspectors where there are critical things.
If you're torquing the connecting rods onto the crankshaft, it says that you've got to do those bolts at 50 something foot pounds.
And the technician will do that.
And there will be an inspector there verifying that he set the torque wrench at 50 foot pounds and it really clicked there, and they stamp the procedure that it was done that way.
I wanted to just add to that, I mean, even more emphasis.
You don't do anything to the vehicle or to the payload unless you have a written procedure.
What is absolutely not allowed is oh, there is a little problem, something didn't work the way it was supposed to in the test.
Let's just fiddle around.
Let's throw a few switches and see what happens.
Well, I mean, that is normally the way you work at things if you're in a laboratory or working on your car.
But the thing is you need absolute traceability.
Because if something later on turns up out of spec or there is an anomaly during flight, you need to be able to recreate everything that was done to the orbiter.
That is why we keep such good records on parts.
Well, you can tell you that story about the tires that you just mentioned.
People keep track of where all the parts of the orbiter came from, the lot numbers.
And so, if any problem turns up, you can actually then trace on the orbiter and see if it affects there.
All the work is done for approved work authorization documents, every step, every touch labor item that is done on the vehicle.
And if you deviate from that procedure, which is approved by engineering, you have to get engineering approval to deviate from it.
The assembly process, the standard work process of putting the vehicle together after a mission, there is over two million verifiable work items for the standard flow.
And that doesn't include modifications or the nonstandard work that comes up, the proverbial glitch that occurs when you're checking out your computer system or when you pressurize this fluid system and the leak rate is higher than the specification allows.
Well, engineering comes in and says OK, here is the trouble procedure.
Here is what you're going to pursue, but you don't do any of that until the engineers write the procedure and give it to the technicians or the console operators that say OK, here is what you're going to do to try to find out the source of this leak.
As Jeff indicated, you don't just start throwing switches and turning valves and say well, let's see if we can find out what's going on here.
It is all pre-approved.
And the reason for that is when you have these reviews that culminate into final review before you get ready to fly, you want to be able to say that this vehicle is assembled per print, that the designer's requirements, the people that you've already heard from that certified that this system will work just fine in flight if it looks like this when we launch it, you're trying to prove that this is what it looks like when you launch it and it is within the certification base and it is per the engineering drawings that we were given and it met all these requirements.
Otherwise, you are guessing that it is OK to go fly the machine.
And you cannot do that in this business.
[AUDIENCE QUESTION] Yes.
Does that seem to create any sort of tension on the engineers feeling like their hands are tied to go out and do stuff?
Well, no, because they are the ones that ultimately say what goes in those two million steps.
And if they want to add more tests or checks or inspections it is within their purview to propose that.
The program, there is a group of senior engineers that are on these what we call control boards that approve the requirements that are implemented at the Kennedy Space Center.
And they can say yeah, that's a good idea or, no, we don't want to change this, go back and give us some more rationale for what you think is a good idea to do to this system.
In that respect, it is probably somewhat frustrating for the engineers because they cannot do anything different from last time unless they can justify the rationale for it.
But that's the way it ought to be.
The launch pad.
The facilities at the launch pad are pretty simple.
These are the same launch pads we flew from in Apollo.
We've got storage facilities for the liquid hydrogen, liquid oxygen.
We try to spend a minimum amount of time at the launch pad because Florida is a nice place to be on spring break but it's a bad place to have exotic hardware sitting outdoors with all that salt, humidity and all the other elements that mother nature throws at you like lightening.
And, I think you can see down here, we have a good lightening protection system.
There is a single mast with two wires that go out about a half a mile from the launch pad.
It makes a good Faraday shield.
And over the years we've been whacked a lot of times with lightening.
In fact, it's a good attraction for lightening.
And we have never gotten energy inside of that Faraday shield to damage any hardware.
However, when we get lightening down there, we tell the workers to stand down and, of course, seek safe haven.
But we have instrumented it.
And it has taken some big hits, but we haven't gotten anything major inside that would cause us a problem from electric.
After you take the Shuttle out onto the launch pad, is it very expensive to roll it back when you find a problem and then bring it back out again?
Yes, it is.
And that's why we have these reviews before we commit it to go to the launch pad to make sure that we're not taking any unknowns or any work out to the launch pad that should rightfully be done either in the Vehicle Assembly Building or further back in the Orbiter Processing Facility.
It is a lengthy process because it takes about a week to hook everything up at the pad and check it before you go into the process of servicing the propellants and doing the hazardous work which has to all be undone if you roll back to the Vehicle Assembly Building.
You've got to make sure you're really ready to go before you head on out there.
Approximately how many times has that happened?
Well, we probably have done it for technical reasons maybe 20% of the time.
For weather reasons like we're worried about a hurricane or something like that the same, about 20% of the time.
One out of every five times we've gone to the launch pad, we've had to come back for either a weather problem or a technical problem that either occurred at the launch pad or it's something that happened with a different vehicle in the fleet that may still be in the Orbiter Processing Facility or a components that is similar to one that is on the Shuttle that is at the pad that they were doing testing in the laboratory or doing what we call fleet leader testing at one of the NASA facilities.
And they said, you know, there is an inherent flaw in this auxiliary power unit and we have to change the ones that are in the orbiter because they were manufactured at the same time or they have the same component on them and you cannot do that work out at the launch pad so you've got to roll it back.
So the hardware at the pad may be just fine from the standpoint that you followed your processes and they checked out great in the previous part of the campaign.
But an offline issue can cause you to say no, we've got to roll back and go change out this hardware or fix it or further test it before we can commit it to go fly.
We have a big water tower out there also which I will talk about later.
That is our sound suppression system.
But basically a month is the maximum time you like to spend out there.
We used to put a lot of payloads in the Orbiter Processing Facility when we were flying laboratories and that sort of thing.
Space Station hardware, we install all that out at the launch pad now.
And we test all the connections at the new facility.
And we have a simulated launch count with the astronauts.
And this is a tradition that goes back to the Mercury program.
They bring the astronauts down to the Cape.
They go through a day, two days of training, emergency egress, familiarization with the hardware, And then we have a simulated launch count without the propellants and all the hazardous stuff just to, if nothing else, remind the launch team that this is more than just a machine.
We are going to fly people in this thing.
And this occurs roughly two weeks before launch.
And it is a good readiness test for the whole team.
We're a couple weeks before flight.
The crew is in town.
This vehicle is going to look the best it ever has.
And we do a simulated launch countdown to within a couple seconds of liftoff.
And then we do a simulated abort with the safeing of the vehicle and the crew gets out.
And I will just add something.
That is always something the crew looked forward to, first of all, because you actually get to get in the vehicle which, despite the fact that we have pretty good simulators in Houston, there is nothing like actually crawling inside the shuttle that you're going to fly in.
And there is a lot of safety training that you do down at the Cape which is really kind of fun.
You don't have any pictures of the launch escape, the pad slide wire basket?
No, I don't think so.
There is a requirement that you need to be able to get off the pad quickly if there is a launch emergency.
And, of course, on the real launch day, unlike the simulated countdown where the pad is crawling with people, on launch day there are very few people, even when you're getting into the vehicle because it is fueled.
And then everybody else leaves.
So they have this big slide wire where if you have to get out of the orbiter in a hurray you go out and you jump in the bag, hit a little guillotine and it cuts you loose and you slide all the way down.
Actually, in the early days, the astronauts kept saying we need to try this out.
And they said no, you cannot do it because it hasn't been man rated.
We said wait a minute.
Well, it's rated for emergency use only so you cannot try it out.
And then after Challenger they insisted that somebody actually ride down in it.
Not every crew.
Just one person tried it out.
But the other thing that you do, once you get down to the bottom there is an underground bunker that you run into.
And either you stay there and wait for help or they have these armored personnel carriers sitting around there with little tanks which you get in.
And you've got to be able to drive your tank.
They have a breakout section of the fence so you drive the tank through the fence out to a helicopter pickup point.
So we all have to go out and learn how to drive the tank.
And we drive over the sand dunes and everything.
And then also they have a big pool which they light on fire.
And they give you a big fireman's hose and they show you how to make your way through a fire by squirting a water path in front of you.
All the little things you dream of as a kid, to be a fireman, drive a tank.
So it's good fun.
But it is part of the extensive safety procedures that they have.
And all these things periodically they have to exercise.
They do go through, every once in a while, a disaster drill down at the Cape where they simulate an emergency where they have to pick up the crew either outside the launch pad or either out in the ocean.
And you've got to keep people at full operational readiness.
That's one of the aspects of training.
We train for the anomalies, the nonstandard things.
The standard work, they do so much of that anyway you don't have to train for that.
You don't have to train to put a payload in the payload bay.
You don't have to train to power up your system and run your system checkout because that is a repetitive thing.
But the nonstandard stuff, when things go wrong, that's where the emphasis of the training is.
And, like Jeff said, it's a lot of fun.
It's fun for the launch team, too.
We have a control center at the Cape, and it is called the Launch Control Center.
You would think this is where you launch it from.
Well, you do that.
But all of the other work that is done in the preceding three months is controlled from the Launch Control Center by test conductors and engineers who wrote the procedures and the software that implement all the requirements.
And they control the activity on the orbiter or the tank or the boosters when it is still back in the offline facilities or the Vehicle Assembly Building or out to the launch pad.
They do it from the Launch Control Center.
We automate what we can from there.
But for the manual activity -- And a lot of it is manual because regrettably you cannot do all this refurbishment on the shuttle with automated systems.
It wasn't built that way.
It is all managed from out Launch Control Center with old computers but with a great team of people.
The software we use in the computers, I am not going to go into a lot of detail because it is really old stuff, but it works.
It is like the software program that the FAA uses to track airplanes.
It is hacker-proof.
There are no external interfaces outside of the Launch Control Center firewall.
It works just fine.
And that is where we do the management of our day-to-day operations from our Launch Control Room.
And we call it the LPS Launch Processing System software.
We've tried to change it a couple of times over the years.
Buy new hardware.
We have supplemented it with laptops so that we can do more human engineering displays, but the computers are the same ones that we bought back in the late '70s.
But then so are the computers that are on the orbiter that fly the machine.
So it works.
This is a bit repetitive, but I want to go through it again to make sure to emphasize what the role of the engineering is.
The designers that you've heard from, they certified their design.
And they did extensive testing, just like Detroit does on a car on the door latch.
And they've tested that door latch and said, well, this thing out to be good for 250,000 miles or 20 years or so many gazillion opening and closings of the door before it wears out and you've got to replace it.
Well, they did all that.
And to make sure that that hardware is still within the certification base, they developed requirements, sent them down to the Cape.
We put them in the procedures, either manual procedures or software to implement those requirements.
And the team, United Space Alliance the contractor and the NASA engineers will certify at each of the milestone reviews and the process of moving the hardware out to the pad they've met those requirements.
And they did that either by participating in the activity or by reviewing the test data.
And that allows us to sign a certificate of flight readiness a few days before launch that all the requirements have been met.
When we get into launch count, we take in a subset of those requirements that when everything is powered up and ready to go fly all of these systems that are active ought to look like this.
The voltages should be between this upper limit and this lower limit.
The pressure should be between here and here.
These indicators should show that they are open and these indicators should say that this latch is closed or the valve is closed or whatever.
And we put that in what is called launch commit criteria.
And that is the acceptable limit for the performance of that hardware when you're at that point which is nine minutes before launch when everybody commits that they are ready to go fly.
And all those requirements, again, are implemented by procedures in software.
And the three days of launch count have roughly, these are numbers, but about 500 requirements.
There are thousands of measurements associated with those requirements.
And most of the looking at those measurements is done by computers, obviously.
But you have the engineer involved because if it is not within limits you turn to the engineer for what is wrong.
Is there any chance to go fix this or do we have to change hardware?
Do we have to do trouble shooting?
So it's not a hands-off operation, obviously.
The structure of the launch team, the launch director is shown in the middle, but that doesn't mean he or she is the only one with a go or a no-go button.
The people the implement the requirements, there are roughly 150 of them in our Control Center, and they are the ones that provide the go, no-go for all the subsystems both flight and ground that everything is operating within the limits.
They report that to a team of test conductors, NASA and contractor, and that's reported to the launch director.
We are not along, obviously.
We have an engineering support area, which are all the hidden senior engineers that you don't see on television.
And there is a room full of about 150 to 200 of those.
And they are looking at the performance of these systems as they are activated in launch count.
And they've got all the trend data, the previous history of it.
And they're looking at things like, well, all right, everything is in limits, but is it within family?
The last time the system performed the voltage with this or the pressure was this.
Well, it's still in limits.
But it's not the same as it was before so we go look at it.
Well, that's for this group of engineers over here.
These guys and gals are running through their procedures.
And, if everything was within limits, we just turn to the next page and go onto the next one.
And they also have the emergency procedures.
If something bad happens these are the people that will implement the emergency procedures.
This offline support, they're out there.
They're the senior people, they've been there before, they've been out there who knows how many years and they're asking questions like that didn't look right or we missed that step, go back and check it again.
That's what they are for.
The mission management team over here, these are the people that do the certificate of flight readiness.
These are the folks that stand up in the readiness reviews and say I gave the Cape folks a good set of requirements.
There is nothing that happened with the hardware that they're responsible for.
And all those offline facilities are on previous missions.
That shouldn't have any cloud over this mission.
They are responsible for certifying that.
And they do that through a structure of the mission management team.
You've got, of course, the flight director with flight rules and the flight team that they look at during the launch count process.
And you will hear more about that from Wayne Hale.
And they, obviously, have a go, no-go input to the launch director in the decision process also.
We have an integration activity.
They system engineers are each responsible for, you know, they have boundary conditions on what they are responsible for.
It is these pieces of hardware.
It is this wiring.
It is these actuators or latches or whatever.
But a lot of that stuff fits together.
Integration is a technical term for making sure that this person talks to this person and that they are hooked in with this one.
That activity is done by a console in the Control Room, again, with senior engineers who actually manage all the automated software that runs the last couple hours of launch count that really launches the vehicle.
And they are tied in with that.
You also have the more subjective stuff.
We launch on a public range.
Public safety is obviously an issue, so the range has to make sure that it is safe to fly from a public standpoint, there are no boats in the launch danger area, no airplanes and that the weather meets the criteria of being able to track the vehicle should it go off course.
And, of course, the payload has to certify, particularly if there is an active payload, that all of their launch commit criteria is met.
And last, but certainly not least, you have safety oversight of all of that activity.
And what I don't show on there, of course, is the flight crew, but they are obviously in communication with all of these people.
So that is the network.
But everybody out here has no-go authority.
Anybody can come on the Net or press their switch and say I've got a problem, we're not going anywhere.
And it is the launch director and the launch team's responsibility to make sure that problem is addressed, hold at the convenient, graceful to stop all the activity hold point until the issue is resolved.
The real job, and I'll get into this later, of the launch director is to say no when everybody else wants to go because, as you get further into the launch count, the "launch fever" process sets in.
It is a natural thing, it seems to be.
People want to go.
I want to ask you about that a little bit, I know we'll get into it later on, but flight rules and the launch decision, there are flight rules that say you must scrub if these conditions are not met.
And, yet, there is a tendency to allow judgment to come in.
Let's take an example of a limit on crosswind at you landing to the return to launch site.
You see it go above limits, but the rules strictly says no, you cannot go.
Talk about that a little bit.
That is what we use the mission management team for.
And you bring up the weather example.
Most of this stuff is pretty straightforward.
It is either the voltage pressure temperature is in limits or out of limits.
A person can question that and say I don't like the trend, and you've got to stop and clear the air if they say I want more discussion on this, I want more data review and that sort of thing.
If it is a discussion like that involves activity outside of the Control Room beyond the purview of the console operators that are running through the procedure, we rely on the mission management team to manage that activity so the console operators can concentrate on their launch commit criteria and their procedures and their software.
Something like crosswind limits, if there is a debate with the flight director and the weather people, we hold the clock and say mission management team, that is yours.
You manage it, and whatever you have to do to resolve the decision, we'll scrub if need-be, we'll hold if need-be, or if the community can get comfortable that these crosswinds that are peaking occasionally out of spec, that it's OK to go fly anyway, then we will wait to hear from the flight director that that's OK.
We don't insulate the Control Room from that, but we don't want to burden the control center with work and what is an offline issue.
And it is up to the mission management team to disposition that.
And you bring up weather because weather is one of the few things that there is really a lot of judgment still involved.
Is the weather good enough to go fly?
And it is probably the only thing that is in that category because everybody has an opinion on the weather.
And Florida weather is dynamic.
And usually, unless it is wintertime, there are clouds.
And there is always a tendency, if you've held for weather or the weather is marginal, to hold longer to see if the weather gets better.
And the real issue for the launch director is to try to convince people, if the weather is good enough, that this is good enough.
And, yeah, you could wait longer for it to get better but it's good enough.
We're running out of window time.
All the time we sit here on the ground with all these systems humming along, the probability of some glitch or something coming up to scrub you is increased.
And just as a personal note, when I was I the Air Force, I was a weather forecaster for Missile Operations.
And I felt that that was a waste of an engineer's time because I was a graduate engineer with my diploma sticking out of my pocket.
I wanted to go launch missiles and rockets, and the Air Force said Lieutenant Sieck is going to be a weather forecaster supporting Missile Operations.
I endured that for a couple of years and put all of that experience in my hip pocket.
And I didn't need it until I got into this launch director job.
And I spent many hours talking to the weather people who provide the forecast for the launch going over all of the data and the technical aspects of the weather situation to see if there it was prudent to sit here any longer and wait for the weather to improve or whether it was better to scrub, get these guys out of their uncomfortable suits and send them back to crew quarters and recycle for the next day.
And they won't necessarily scrub, but you have to disposition their concern.
And you will hold until their concern is taken care of.
This is unrelated, but you know how the Russians, in going out to the launch pad, use a rail track and they have the rocket flat?
Can you do that with the shuttle?
Instead of stacking it like this and then rolling it out, would it be better, easier?
Well, the problem with that are the solid rocket boosters.
They weight approximately, when assembled, a little less than three million pounds a piece.
And the joints that caused the Challenger accident are critical.
And I don't know if they went through the technical aspects of the design, but these things are 14 feet in diameter.
Each segment weighs 300,000 pounds and they have these two cleavaces that come together with O rings in there.
And what caused the Challenger accident was there was rotation in these joints to the point where the O rings, because of cold weather, weren't making a good seal and caused a gas path.
And 1200 degree temperature.
Exhaust plume just cut through the metal once it found a path.
So you want to maintain the integrity of these joints.
If you stacked them horizontally, and that could be done, and then you lift this thing vertically, the joints aren't certified to maintain their integrity doing that.
Plus, you'd need some kind of huge crane to pull this 3 million pounds from horizontal to vertical.
I think it's something that could have been done, had it been built into the design at the beginning.
And the Russians have a tradition of doing this.
And so that is the way they design their rockets.
For whatever reason, we never started that way.
From the very beginning, our rockets got stacked vertically.
And that's the way all of the American rockets have been designed.
[AUDIENCE QUESTION] The launch director, whether it is flight worthy, actually, that mission management team is responsible for the flight worthiness because they encompass not only watching what you did in launch count, but the certification of all this hardware before it even got to the launch pad.
And, ultimately, the launch director is responsible for conducting an orderly launch and making sure that this mission management team doesn't have any issues, that that engineering team doesn't have any issues, that the console operators have really completed their procedures.
That is their responsibility.
And, if there is any fuzz on that, so to speak, even though console operators say they're go, and the mission management team says they're go and flight says they're go, if there is any concern about that, it's the launch director's job to say no, we're not going to go fly today.
We are going to give you another day or two days to do more homework, look at more data, have more discussions.
We're going to scrub today, even though everybody may say we're go.
And that has happened before.
And I always wondered, after not too many times, when I've said no, when everybody wants to go, whether I'll get the phone call saying well, I'm glad you made it safe today, Sieck, but you don't have this job tomorrow.
Well, I never got that call.
The process, not to bore you with it, but since it is an operation, normally we try to launch in the middle of a week, Wednesday or Thursday.
We usually take the weekend off before that to clean up everything and give the launch team a rest, so we power up everything.
And that's the way we start the process.
And that's a couple of shifts worth of work to bring up the ground systems, the flight systems, make sure that all the avionics hardware really comes to life.
And we put in a no work hold at the end of that to take care of any problems.
If that all goes well then we get into the more critical activity that has time constraints associated with it.
We load the fuel cell.
That is the power plants and the orbiter reactants, liquid hydrogen and liquid oxygen, because they have limited life.
Even if you don't have the fuel cells activated that stuff boils off and you lose it and that could count against your mission capability.
And that is a hazardous operation also.
And then we put in another no work hold at the end of that.
And then we have a period of time where we bring up the final systems, put in the time critical storage, particularly when we're flying laboratories and you put in plants or critters or whatever it may be either in the crew module or the payload bay.
And the last 12 hours down here is when you really have got to make sure this is a real decision point right here.
Are we going to load the external tank, put a cycle on the tank?
And we know that putting liquid hydrogen and liquid oxygen, and there is roughly half a million gallons in a tank, puts stresses into the hardware.
You really don't want to do that unless you're pretty sure you're going to launch that day or that night.
We have a meeting before that to make sure the weather looks pretty good and that the mission management team is not working any offline issues.
And that has happened before.
A war story for you.
In the early '80s, in one of our missions, we assembled the mission management team to decide whether or not we ought to go load the external tank and go fly.
Everything looked pretty good.
And the orbiter project manager said well, I need to make you aware of this.
I just got a telegram from the manufacturers of the tires.
And the manufacturer said they were inspecting a lot, a lot of the tires of which we have two of them from the same lot on the Orbiter, and they found some blems on these tires.
And they don't know whether it's a manufacturing flaw or an aging defect.
And we don't know whether it affects the integrity of the tires.
But since we knew you were launching tomorrow, we thought we would share this with you.
Warm regards, BF Goodrich, I think it was at the time.
Well, the mission management team threw on that on the table and said hey, we've got to go work this.
These people, we don't know, they haven't said they're not certifying the tires.
They just said they've got a problem.
Well, that's fine.
The mission management team, go work that and let's talk about whether or not it's prudent, under these circumstances, to go load the tank with some probability that we get all the way down to T minus nine minutes and you folks are going to say hey, we need to look at more test data on these tires or have more discussions with BF Goodrich on whether or not we go fly today.
Well, you don't want to load the tank and go through all that if that's what you think the situation is going to be eight hours away from that.
And you've put a cycle on the tank, you've used up a lot of cryos that you cannot save because of the heat leak.
Plus, your launch team has put a cycle on, depending on the time of day, they've been up all day, all night or whatever.
Those are the kinds of issues that you need to make sure get worked.
And the mission management team has to handle that, but the launch director has to decide just go work that for another day or two.
We're not going to load the tank.
Or, what's the promise of this coming to fruition, the possibility so that it makes sense to go try to fly?
What did you do with the tires?
In that case, we went ahead and tanked.
The subsystem manager for that, which you may have already heard from, went back to a lot of the test data that they had run on the tires, that they had been fairly recent.
Langley, I think, was involved in it.
And they said we don't have to worry about those blems being a problem for the tires that are on the Orbiter which, by the way, you don't have any access to at the launch pad.
If we were going to change the tires, you would have to roll back to the Orbiter Processing Facility.
And that would be a couple of month's worth of impact to that flight.
We ended up not doing anything with the tires, but it had the mission management team really busy for about eight hours.
And this process takes about three days.
And we have refined those procedures, as you might expect over the years, to make this as an efficient and repetitive a process as we can.
In addition to minimizing the hazards to the people and the equipment.
Before we got to terminal count, usually we take like a two minute break.
This is a good time to do that.
Terminal count phase.
After the tank is loaded, half a million gallons of liquid hydrogen, liquid oxygen, and there are no leaks, and we have leak detectors all over the vehicle, and we've scrubbed a lot of launches because of leaks, we give the crew a go to come on out.
And a crew of seven or eight people, it takes an hour and a half to get them all in and get them connected, check their communications and that sort of thing.
Roughly three hours before launch we bring them out, we get them all connected, we close the hatch and we verify the integrity of the crew module.
And clockwise we're down to T minus 20 minutes, but if it's a rendezvous window, and most of them are now.
The launch window is only five to ten minutes long so we have to be in phase with the same plane that the Space Station is in and we've got to get in phase with being able to catch the Space Station or have it catch us.
The launch window, we don't have enough energy in the Shuttle to steer to any orbit we'd like to.
The launch window is only about ten minutes long, so we put in a long hold at T minus nine minutes for those kinds of missions.
It is the equivalent of the two minute timeout in a professional football game.
Everybody has a chance to review their information, discuss their strategies, if it's a long launch window, if it was just a lab mission then we only wait for ten minutes at this T minus nine mark.
But, if it's a short window, we set up the clock such that we hold there for almost an hour.
And then, as you saw on that previous chart, we do a poll of all those people and make sure that they are all still go and they're not working any problems or issues.
And then we start this automated software program that looks at, in addition to the systems engineers and their consoles looking at all their information, we have an automated program called the Ground Launch Sequencer that looks at all the measurements, all the parameters to make sure that they are within limits.
And it issues all the commands to the vehicle and the ground support equipment.
And we did that so we could manage the repeatability of that process.
And you had to take the human factor button pressing out of the process as much as possible.
And there is not a lot of work that goes on in there.
And there is very little manual work.
You get five minutes.
The orbiter access arm, their ability o get out of the vehicle in a big hurry is retracted.
It takes two to three minutes to pull that back, but we can get it back up to the vehicle in less than 30 seconds which is about the amount of time it takes them to unstrap and get out of there anyway if there's a fire or something bad like that, that they have to do an emergency egress.
They start the propulsion units on the Orbiter that pressurize the hydraulic system.
Just like on an airplane, there is an automated test of all the aero surfaces like they do on the end of a runway.
But this is done by the computer and it wiggles the elevons and the rudder and the main engines to make sure all of that works.
And we also start the conditioning of the propellant at the engine interface.
And they already had the discussion about the main engines.
You have to make sure that the quality of the fluid in the external tank and the quality of it that's right there at the injector of the engine is within the temperature and density parameters that the engine was certified at.
Because we have to drain the propellant out of the fill lines.
And, once we've started doing that, that propellant starts heating up right there on those valves that are going to go into the injector and start the main engines on the Orbiter.
That limits our ability to stop and hold after that.
We are good for four to five minutes after that starts.
And that process starts at T minus five minutes.
After that point in time, we are either going to launch or scrub in the next ten minutes.
And then we pressurize the oxygen tank and we pressurize the hydrogen tank, but anywhere in this period of time we can stop.
If an engineer or the automated sequencer says something is out of limits, we will stop at those milestones and disposition that problem if we can.
But, per our rules, we won't waive any requirements at that point.
Even though an engineer may have a perfectly good explanation as to why this temperature, pressure or whatever, our limit is not being met, if the launch commit criteria, the launch rules says no, it has to be like this, then we're not going to go launch that day.
After 31 seconds, the solid rocket boosters come to life.
The propellant doesn't ignite, but their hydraulic system is powered up.
Their avionics systems are powered up.
Their engine nozzles are checked again in an automatic sequence.
And at 10 seconds, if everything looks good and all these measurements that our Ground Launch Sequencer has been looking at, we send up the computer data bus a go to the onboard computers that said all of our stuff has been satisfied.
Now, we're still looking at a few things.
We're looking at those bolts that are holding the solid rocket motors to the launch pad.
And the ability to blow the nuts that are holding them, should that system fail, we would send the cutoff to the onboard computers.
Regardless of where the engines were in their startup sequence, the three main engines, everything will stop.
And hopefully it will gracefully come to a stop and we will safe all the systems and go from there.
But that is the final handshake that is preplanned between flight and ground is that ten seconds when we send a command up to the computer saying we are all go.
We can still shut you off, and we can shut you off manually, too.
If a console operator sees something happening that they think could compromise the safety of the launch, they can call for a cutoff.
And we have a switch that says cutoff.
It sends a command to the onboard computers, and it will stop everything within 10 to 20 milliseconds.
But you don't like to put humans in a position to have to make that call.
And that's why all of the critical stuff is done by the computers.
And I mentioned the Ground Launch Sequencer.
That's a console and operators in the back of the Control Room.
And they are the ones, by the way, that determine nobody presses a button for launch.
A woman in the back of the Control Room types the liftoff time into the computer when the launch director tells her, hey, look, it looks like we're going to come out of our T minus nine minute hold at this point in time.
So here is the liftoff time.
Put that in the computer.
And after that it's a hands-off operation.
Can I ask, are these commands being physically sent, I mean they're hard line cable?
It's what we call a launch data vest.
It's a cable that is connected from the computers in the Control Room through the liftoff umbilical into the Orbiter computers.
It's a data train.
And when is that connection broken?
The onboard computers will check the health of the main engines, and if the three main engines are within all of their operating parameters, the turbines and the pumps and the temperatures then it sends a command to those eight bolts that hold the solid rocket boosters to the mobile launch platform and the bolts that hold the external tank vent arm to the tank.
And the two Orbiter umbilicals that says fire the nuts, we're going to go.
And that is orchestrated by the onboard computers.
And when that happens you're flying, you're going.
The command to ignite the igniters on the solid rocket motors is sent at that time also by the onboard computers within a couple of milliseconds.
So all that happens at once.
And it's an onboard automated thing.
I talked about the human factor that repeatability is important.
And I mentioned I gave the engineers, you know, I always threw rocks at them because they always want to change things.
It is the launch director's responsibility to manage.
They're the owner, so to speak, of the launch count procedures, the 5,000 pages of documentation and the 500 or so software programs that are executed in the launch count process.
And we discouraged changes to that for obvious reasons.
Unless there is a modification to the flight hardware or the ground hardware or we found something in our previous launch attempt that said we need to fix this or there's an opportunity to increase the margins, we don't change it.
We try to maintain the procedure and the hardware the way it always has been.
And it essentially is.
We do training, obviously, but we train for the nonstandard work.
Just like in the flight team, we have a simulation supervisor that throws diabolical failures out there and we have computer programs and into the control center we bring the launch team, power up the consoles.
And their displays look like the shuttle is at the pad and it is getting ready to launch and we throw failures at them to test their reaction and to find bugs in our safing procedures.
And we've done that over the years.
I think the next generation of vehicles will probably have more automation.
And that will probably be a better thing but you will never take the console operator out of the Control Center either, the Launch Control Center or the Flight Control Center.
You need the person there to disposition the problems, but predictability is what we're always after.
And stable procedures, be they flight or ground, give predictable results.
And that is why we automate.
And we control change, too.
I talked about the engineers.
I didn't mention, I don't think, technicians and inspectors.
But when a technician stamps a procedure that I torque that bolt and the inspector stamps that same procedure and said I saw him torque that bolt, that is their warranty that that event really happened.
And that paper trail and that warranty is very important.
When an engineer signs a procedure that tested the Flight Control System that said I reviewed all the data, didn't see any glitches, everything met specifications, that signature is his or her warranty that the procedure ran correctly and that they understood the requirements that were implemented by that procedure.
And the requirements were met.
And that is important.
Our whole concept of launch readiness is based on people being responsible for their system, be it the designer that signs it that says there is nothing going on offline, there are no faxes out there that say there is a cloud over the tires.
Or, my last test of the main engines at Stennis, there was nothing there that says these three engines that are on this orbiter shouldn't work just fine.
And they do that in a flight readiness process.
That is their warranty and responsibility.
And we hammer this home.
And it probably comes from, I had this quote in my office for years and I am going to bore you with it, and maybe some of you can guess where this came from.
Responsibility is a unique concept.
It can only reside in here in a single individual.
You may share it with others, but your portion is not diminished.
You may delegate it but it is still with you.
You may disclaim it but you cannot divest yourself of it.
Even if you do not recognize it or admit its presence, you cannot escape it.
If responsibility is rightfully yours, no evasion or ignorance or passing the blame can shift the burden to someone else.
And this is what we push home.
And, by the way, this quote was Admiral Rickover who was testifying before Congress after the first nuclear submarine accident.
And he was saying I am responsible.
It is my program, it is my submarine, I am responsible.
Even though somebody else was commanding it, it doesn't matter, I am responsible.
And we impress that on the managers, the engineers, the technicians, the inspectors, you know, you're important.
And the work you do is important.
And when you sign or stamp that procedure or give a go on the Net that is your warranty that everything is working and you understand the requirements.
Very important in the business that we're in.
On the chart.
This structure has been in place since the Mercury program.
There is nothing new here.
And the only issue comes in when there is a gray area.
Who makes the call and how much judgment is involved in the call?
And I gave you the example of weather.
Everybody has an opinion on it.
It can be judgmental.
Well, the weather decision is if it is launch weather related it is the launch director's responsibility to determine if it's good enough.
If the flight director has an issue with the return to launch site winds, that ultimately is his or her decision.
The people may get involved in it and they may have offline discussions with the flight director, but it is still ultimately his decision.
And that is the way it has to be because they are responsible.
This mission management team is not responsible for the go, no-go decision on weather on launch day.
It's the flight director for the flight stuff.
It is the launch director for the launch-related stuff.
And it is set up that way so there is no fuzz on it.
Other people can have an opinion, other people can have input, but unless this person up here says launch director we're no go, it is the launch director's decision in that case.
And I will give you an example of that.
Most of you probably don't remember Apollo 13, but Apollo 13 was like Challenger and the Columbia except we didn't lose the flight crew.
But it was just as disastrous an event.
And by all rights the crew shouldn't have made it safety back to earth, but thanks to the heroics of the flight crew and the flight team and the timing of when that event occurred they were able to get back home.
But that was all caused by activity that was done on the ground well before the launch.
And, specifically since I was involved in that, I remember this like it was yesterday.
Of course, part of that is an old age thing.
Stuff that happened 40 years ago comes in loud and clear.
Stuff that happened four days ago, I have a hard time remembering.
But we had a tank in the spacecraft that we ran our simulated launch count.
And back then we put propellants in the vehicle, all the hazardous stuff and detanks.
And we couldn't detank the liquid oxygen out of this tank.
They had the fuel cells on the spacecraft.
And just a little more history.
The tank had been dropped when it was installed back in California, but engineers and managers did some tests and some rationale on saying well, it's still OK to install it.
We got it down the Cape.
We put the liquid oxygen in it and all the rest of the system.
We couldn't get it out of this tank.
And after reviewing it they found, well, the stand pipe in there that is used to drain it on the ground was probably bent or damaged and that's why we cannot get it out.
So the managers and engineers got together for a few days and said turn the heaters on in the tank and vent the liquid oxygen off.
Heat it up and it will boil off as oxygen gas through another loop that takes it through the fuel cells and out vent on the vehicle.
Use the heaters to go do that.
So they developed the procedure and we went on station at night a week after we had the anomaly and turned everything on and turned on the heaters.
And it is fine.
You can see the liquid is turning to gas and it is coming out.
And the console operator in our Control Room said hey, Sieck, I cannot monitor the temperature in this tank anymore.
Well, why is that?
Well, the range on the temperature is from ambient down to minus 300 degrees because it's measured in liquid oxygen.
And it is upper limits.
It started heating up, and now I cannot tell you whether it is 50 degrees or 350 degrees.
I said OK, stop.
So we stopped the test and got all these managers involved, including the people that built the tank and say hey, we've lost visibility.
We're putting energy into this tank that has liquid oxygen in it.
We cannot monitor the temperature anymore.
The pressure seems to be fine.
They said don't worry about it.
There is a thermostat in there that is heat sensitive, and it will open up the power to the heaters if it gets to some limit in there.
Just keep your eye on the pressure.
Make sure the pressure doesn't get too high.
Well, of course the pressure isn't going to get too high because the vent is open.
So we said fine.
We turned the ground power supplies on and let the thing cook for 10 to 12 hours.
And what we didn't know was that the thermostat wasn't certified to the voltage we were using from our ground power supplies because we didn't tell them, although our requirements allowed it, that we cranked the power supplies up to 50 volts.
And why did we do that?
Well, the more energy you put into the tank the higher the heaters get and the quicker the liquid oxygen boils off.
You know, high school physics.
But we didn't tell them that.
And the people that built the tank and built that thermostat had only certified it to the voltage of fuel cell supply which is half that value.
So the thermostat welded together, the points did, and we had continuous power going on in there for eight to ten hours.
It got to 800, 900 degrees in the tank.
We burned all the insulation off the wires to the fans and the heaters.
And, from that point on, we were just waiting for something to happen to make those wires come together.
And if you didn't turn the power on you're going to get a spark in an oxygen tank.
Not a good thing.
Well, that is what happened a day into the mission when they were heading to the moon.
And they turned the heaters and the fans on to stir the cryogenics because they were worried about stratification and zero gravity.
And they got the spark and it blew up the service module.
The point is that communication, we wrote down in our procedures what we did, but we didn't communicate that on a real-time basis.
And, although we reviewed the procedures afterward, as the rest of the team did, the designer who was 1500 miles away who had approved what we did over an intercom system didn't review our paperwork.
You hit that thing with 50 volts, it's only certified for 25 volts, even though our requirements allowed us to use 50 volts.
It is a case where requirements didn't preclude us from launching hardware then in hindsight was flawed.
But, on the other hand, communication, we didn't tell them in real-time.
And, as a result, Apollo 13 happened.
So I learned that lesson early on.
I mean even if you're boring them with what appears to be minutia, you're going to tell them everything and document everything.
And, thankfully, they went through our procedures and said well, there it is.
And I thought we'd get fired.
And actually we got patted on the back for documenting what we had done so it made it easier to zero in on the root cause of the problem.
And, as a result, we fixed that and we were able to fly again in a few months as opposed to having to go through a Challenger or Columbia type of activity.
The human factor.
It is hard to describe what the environment is in the spacecraft a few minutes before launch or in the Control Room.
But there is a lot of tension and there is a lot of energy.
And everybody in there, for the most part, has been awake for the last 10 to 12 hours.
And they came to work that day or that night with all hopes of safely launching the Space Shuttle.
And they really don't want anything to go wrong with their system or somebody else's system.
I mean that's the mood.
Because, if something goes wrong, you're going to come back tomorrow or the next day or the next week.
So they really want to hear this everybody, you know, when they do the polls they go, go, go, go.
And the hardest thing is saying no when everything sounds like it's go.
And there are a lot of times when we've had problems with ground support equipment.
It is old.
I wish we had all new stuff for Shuttle.
A lot of this stuff is the same we used on Apollo.
And, in spite of our efforts to maintain it, it is all out in the corrosive atmosphere at the launch pad.
And that stuff constantly gave us problems.
And we would lose some of our redundant systems and some of our primary systems.
And more than once in launch count, when we'd have a problem out there, the team would come to the launch director through that organization chart you saw and say we've got an idea.
We think if we patch around this and then hook this up to this and we throw these switches and we get this result that it will be OK and we can go fly without this power supply being active or this purge system operating normally and all these other things that we've got out there.
And what you have to do as launch director is say well, just sit back and analyze whether or not -- I mean you rely on your engineers to use their Yankee ingenuity to fix a problem, but you have to assess whether or not they're literally shot-gunning it and coming up with the proverbial quick fix because they want to go home in an hour or so and celebrate a good launch.
Just like these guys want to get out of those uncomfortable suits they have been sitting in for the last couple hours and either get out of the spacecraft or get up there in zero G.
As a launch director, you have to assess well, OK, but is this really the right thing to do or are they working this too hard?
If you gave them another 24 hours to look at the data and think about their approach to a fix would the answer be any different?
And there have been a few times when they said well, that sounds pretty good right now but I am going to give you another 24 hours to think about it so we're going to scrub for the day.
And you can hear the big groan in the Control Room.
And you can just see it going through their heads, well, this guy has lost confidence in us.
I mean we all came to work, we get paid to solve problems and get things done, and he is telling us that ain't a good answer, that ain't a good fix.
So you have to balance that when you're in charge, but the real job of the launch director is to say no when everybody else wants to go.
That is really what you are there for.
Because these guys get tunnel vision.
They want to go fly.
The console operators, things are on automatic in that last nine minutes, and they just hope they don't get that little red or yellow light on their screen that says hey, you better go look at this, it isn't working right.
The last people you want to ask is it OK to launch is the crew.
They'll say yes, no matter what is happening.
And we do ask them, by the way, but it is a formality because we know what the answer
is. Unless he's got a fire extinguisher up there putting out something in the spacecraft, he is going to say we're go.
And that is also why we put in a rule that says after five minutes, for these critical launch commit criteria items, the 500 that you absolutely have to look at and certify, you don't proposition us with a change to that after five minutes because we're not going to entertain it.
If they're out of limits we're not going today or tonight.
It is going to be at least 24 hours.
A long time.
But the team that does it, I mean they are good, they are professionals.
And we talk a lot about the Apollo program and how great things were back then, but I just gave you the example of Apollo 13.
The Shuttle team versus the Apollo team has a much higher degree of difficulty situation to deal with.
There are fewer people on Shuttle, yet it's a more complex vehicle than the whole Apollo system.
The Saturn 5 rocket and the spacecraft were simple compared to the Shuttle system.
And we had 25,000 people at the Cape in the Apollo program flying two to three times a year.
And in the Shuttle program back in the early '90s we flew seven times a year with 6,000 people with a vehicle that is more complex, older.
In the Apollo program our ground equipment was new and all the flight hardware was new.
It wasn't reused.
It was fresh out of the box factory pristine stuff.
And requirements were easy, you know, it had to act and look brand-new or you replaced it.
The Shuttle, because it is 20, 30 years old, depending on which orbiter and the ground equipment which is 30 to 40 years old, there are requirements that allow fair wear and tear.
The engineers are constantly pushed with is this good enough?
These wire bundles are frayed and there are dings in this line and that weld is looking like it has corrosion on it, is it good enough?
Well, they are constantly propositioning where back in the Apollo program you didn't have to mess with that gray area stuff.
It either looked or acted brand-new or you replaced it.
And replacing it was easy because money was unlimited.
As a journeyman engineer on Apollo I was told whatever you need you can have.
System engineers seek more test equipment, you need more technicians to be trained on this over here, whatever you need just ask for it.
Money is not an object.
As a manager on the Shuttle program when I finally got promoted out of the job I liked to launch director, I had to tell the journeymen engineers this is all the money you get for next year.
And be real efficient with this and frugal because next year you're probably going to get less.
And your strategy and approach to issues is much different if you have those two different environments to deal with.
And finally the other thing that Shuttle deals with today versus Apollo is the acceptance of risk.
I mean I already mentioned Apollo 13.
Back then if you made a mistake you were patted on the back and told hey, nice try.
Now, is there anything that can be done to better enhance your probability of success?
Whereas, in the Shuttle program, risk aversion in this country, I don't want to get started on that tangent, is becoming more and more a lifestyle.
We don't want to do things we might lose or we might not win or we may not be successful.
And risk, you know, there is less tolerance to it.
And, when you have a problem like Columbia or Challenger, you get half a dozen boarding parties coming into NASA saying here is what you need to do differently or this is wrong with your agency and this and this and you need to go fix that.
And, again, back in Apollo, it was what do you need to be successful?
And it makes it difficult to operate.
Bob, could you comment upon risk aversion and the launch director and whether there were more aggressive and more conservative people who have had your job?
Well, I would say, before I was launch director, there was one.
And he was more aggressive than I was.
He was the console MANOPS person.
And he would push, and he would push until you would say uncle, I surrender or whatever, and then he would make a judgment call.
I was more conservative than that.
I trained the one immediately after me.
And he was pretty much the same but he got the highly visible job, management say that this guy was good, we'll take him away from here and go put him over here managing this large organization and worrying about budgets and contracts.
That is the same thing that happened to me.
And that's unfortunate, but NASA's strongest suit is not succession planning, unfortunately.
It just isn't.
Are you saying the Shuttle program is too risk averse?
Because you also said that, for example, it is your job as launch director to be conservative and to say no when everyone else is saying yes.
No, I wouldn't say that the Shuttle program is risk adverse.
I would say our society is risk adverse.
And you see some of that permeating into the Shuttle program, but not to the point where I would say it is detrimental.
I mean asking questions, digging in to understand what your risks are is an important thing to do.
And early in the Shuttle program we didn't do that.
We had a lot of confidence in hardware after the first few missions.
It's just like a new car.
You put 5000 miles on it, you get the bugs out of it and you fully expect it to last another so many hundred thousand miles.
Well, that was the same expectation in the Shuttle program.
You fly a few flights and you learn that the tiles really stayed glued to the Orbiter and the thermal control system really responds as it should on orbit and managing the temperature and the coolant leaks and that sort of thing.
And you build up confidence and you say it should work now.
We got the bugs out of it and we can go fly this thing as often as those guys at the Cape can turn it around.
But what they didn't consider and what all these engineers, with all due respect that you heard from that certify their systems, they didn't totally capture the environment that the Shuttle sees over the long period of time in their initial certification.
They didn't capture it.
And that environment includes not only what happens in space or the calendar exposure to just time on some of these systems that have soft goods in them, for instance, O rings and that sort of thing, but it is the environment of the Cape with people constantly removing hardware, disconnecting things, moving wire bundles out of the way to get access to this or that other component to implement the requirements that were levied on the Cape to do the tests and inspections.
And they are very invasive, and collateral damage is a way of life when you're crawling around in the Orbiter.
To put in policy that tries to compensate for the fact that the certification didn't capture that environment that this hardware has seen for the last 20 to 30 years, to reduce the unknowns involved in that, that is the right thing to do.
You're trying to reduce the risk of reflying a very complex vehicle.
So that is fine, but you take that to a limit.
I mean you can talk yourself into never flying which would be pretty easy to go do.
And then this recovery after the Columbia accident, that was starting to happen.
The managers, because they had this investigation board report that says NASA, you've become complacent and overconfident.
And, by the way, in my opinion that is somewhat of a bum rap to say that NASA in general, and particularly the Shuttle program, had fallen into that mode.
But, to have the pendulum swing the other way to understand what your risks are and then you can have knowledge on whether you decide to accept it or not, that is a good thing.
What didn't happen after Columbia, but we did do after Challenger, is after we understood the root cause of the accident and some of the other factors like communication that we're part of that, reviewed all the systems and all the requirements, the managers said look, we're willing to accept some risk.
You people who are responsible for these systems and these processes at the Cape, you need to quantify that for us and bring it to us.
And we will either say OK, we won't accept that, you go back and change your system or change your limits or do more tests or whatever, or we will accept it.
We will say we'll accept that risk, management will accept some risk.
That happened after Challenger, and we were able to fly again roughly two years later after we recertified the design to the solid rocket motors.
That didn't happen after Columbia.
And you can speculate whether or not we could have flown six months ago or a year ago or we should have waited longer to fly because we still had a piece of foam come off the tank.
You could debate that.
But management never acknowledged that they were willing to accept some risk after Columbia.
They did say tell us what the risk is but we really want you to crank it down to zero.
We don't want to take any risk, unlike Challenger.
Now, is that bad?
See, what really happened with Columbia was the mission management team's discipline had eroded.
There were communication issues.
And, if you're a student of that investigation, they didn't grade out very well.
But the investigation said there is complacency and overconfidence in the system.
Well, having been part of the system prior to my retirement five years ago, you never found any complacency or overconfidence with the engineers, technicians, inspectors or the managers that I knew.
Now, they weren't as smart as they thought they were or, in some cases, as smart as they needed to be.
But that is not the same as complacency or overconfidence.
Because the latter to me implies an attitude problem that I've got it, I know it all, I am smart as I need to be.
And, for the most part, you know, there were some people I didn't like because I thought he or she was arrogant or something like that, but I never sensed an attitude problem with the Shuttle team.
From the technicians, many of which I knew and grew up with, to the top managers in the Shuttle program, I never sensed that.
But were we not as smart as we thought we were or as smart as we needed to be?
Were there signs that there was something going on here that you ought to go fix?
But, again, they didn't just blow off, so to speak, these problems we were having with foam on the tank and these other systems which, to me, would have been complacency or overconfidence.
We're just about out of time.
I've got two announcements.
I've been working with the Stellar people.
As you know, some of the pdf files that I've tried to load didn't load.
They cannot really figure out why, but I have sent them the files, they've loaded most of them on so hopefully almost everything is there.
We're continuing to work at it, but hopefully all the things that you need access to you can find.
I said there were two things, but that's the only one I remember.
Well, let me close with one note then.
I am glad you're here.
Thanks for your choice to pursue an engineering career.
Doing things that are hard, I have found that in my travels that unfortunately there are fewer people deciding to pursue the tough curriculums and the occupations that require a lot of work and have some risk associated with them.
But it is encouraging to see people in graduate school that want to do what I did.
I am not sure how much engineering I did in my 40 year career.
I did a lot of managing and a lot of other assignments, but I am sure you can accomplish a lot.
I had a grandmother who was born at the end of the 19th century.
And, when she was your age, there weren't any computers, airplanes, cars, TV and that sort of thing.
Yet, in her lifetime, she got to see her favorite grandson, Bobby, help put a man on the moon.
And I don't know what can be done in your lifetime, but you can make great things happen with the career you have chosen.
So thanks for choosing, really.
Bob, thanks very much.
This is one of over 2,200 courses on OCW. Find materials for this course in the pages linked along the left.
MIT OpenCourseWare is a free & open publication of material from thousands of MIT courses, covering the entire MIT curriculum.
No enrollment or registration. Freely browse and use OCW materials at your own pace. There's no signup, and no start or end dates.
Knowledge is your reward. Use OCW to guide your own life-long learning, or to teach others. We don't offer credit or certification for using OCW.
Made for sharing. Download files for later. Send to friends and colleagues. Modify, remix, and reuse (just remember to cite OCW as the source.)
Learn more at Get Started with MIT OpenCourseWare