NASA vows to correct problems that led to dangerous EVA leak
02/26/2014 05:11 PM Filed in: Space News | International Space Station
By WILLIAM HARWOOD
CBS News
A fundamental misunderstanding of how water behaves in the cooling system of a spacesuit operating in weightlessness, combined with institutional complacency and a perceived pressure to avoid delays, resulted in a frightening, potentially fatal close call last summer when water flooded a spacewalker's helmet outside the International Space Station, NASA managers said Wednesday.
With his eyes and ears covered by a growing blob of water -- and faced with the counter-intuitive but very real prospect of drowning in the vacuum of space -- European Space Agency astronaut Luca Parmitano made it back to the safety of the station's airlock in the nick of time, assisted by fellow spacewalker Christopher Cassidy.
The July 16, 2013, incident was the most serious known incident ever encountered during a NASA spacewalk, and the space agency launched an extensive, wide-ranging mishap investigation to find out what went wrong and what steps need to be taken to prevent a recurrence.
The investigation board's report was released Wednesday, blaming the leak and subsequent events on a complex combination of factors that included a misunderstanding of the subtleties of the suit's operation in weightlessness to more troubling institutional issues that, while not intentional, could lead to similar problems in the future if uncorrected.
"While I am concerned about ensuring this particular incident does not happen again, I am especially concerned about cultural factors that may have contributed to the event," NASA Administrator Charles Bolden wrote in an agency-wide note to employees. "In our exuberance to get the job done, we may have allowed ourselves to accept the commonly accepted causes for small anomalies.
"We have a responsibility not to move on from any abnormal situation until we understand it fully or have suitable mitigations to prevent it happening again. Our work both in-house and with our industry and commercial partners should entail diligence in assessing risk and commitment to ensuring mission safety."
The mishap board concluded the leak was caused by a subtle water chemistry issue in the cooling system of Parmitano's spacesuit. Contamination built up and blocked a filter, allowing water to make its way into a line that feeds air to the helmet. The source of the contamination is not yet known.
A detailed review showed the leak actually began near the end of an otherwise-successful July 9 spacewalk. Parmitano and fellow spacewalker Christopher Cassidy initially suggested the water probably leaked from Parmitano's drink bag. Engineers on the ground came to the same conclusion and cleared the astronauts for a second spacewalk, EVA-23, on July 16.
Just 44 minutes into a planned six-and-a-half-hour excursion, Parmitano ran into a pair of problems. A carbon dioxide sensor in his suit suddenly stopped working and shortly after that, he noticed the back of his head felt unusually wet.
A CO2 sensor failure was not considered a major problem. In this case, flight controllers said they would monitor suit telemetry on the ground and keep tabs on how Parmitano was feeling. There were no indications of an actual CO2 buildup.
But as Parmitano pressed ahead with work to route cables on the outside of the International Space Station, he noticed a water buildup at the back of his head where a helmet air duct is located.
"I don't understand where it's coming from," he said.
"It has to be (your drink) bag," Cassidy replied. "Can you suck it dry?"
A few minutes later, Cassidy peered into his crewmate's helmet for a closeup look, saying "so that stuff on your forehead is not sweat?"
"No it's not."
The leak quickly grew and water began to creep around the side of Parmitano's head into the forward part of the helmet.
"I can still hear perfectly, but my head is really wet and I feel it increasing," Parmitano said. A few minute laster, he wondered again, "where's it coming from? It's too much."
"I don't know. It's a lot," Cassidy agreed.
"Now it's in my eyes," Parmitano said.
In the absence of gravity, water tends to pool in blobs and inside the cramped confines of a space helmet, a large amount of water could trigger uncontrolled coughing and, possibly, even drowning.
By that point, flight controllers assessing the problem at the Johnson Space Center in Houston knew the spacewalk could not continue. In fact, it needed to end as quickly as possible. The astronauts were told to return to the station's airlock as quickly as possible, a process that took about 40 minutes to complete. The excursion ended at 9:29 a.m., just an hour and a half after it began.
The astronauts later disassembled the suits and helped engineers on the ground carry out an extensive series of tests to pin down the source of the leak. They quickly concluded the drink bag was not the culprit. Why the ground team went along with that hypothesis in the first place was discussed at length by the mishap board.
Institutional complacency appeared to play a role in the incident, the board concluded, along with a perceived pressure to keep station science on track and not to raise questions or concerns that could trigger delays. The leak first appeared during the July 9 spacewalk, but the board concluded that spacewalk engineers jumped to conclusions when they blamed it on the drink bag.
"The ISS Program must place a strong emphasis on performing utilization with the ISS; it is in fact the very reason ISS exists," the board wrote. "However, the strong emphasis on utilization was leading team members to feel that requesting on-orbit time for anything non-science related was likely to be denied and therefore tended to assume their next course of action could not include on-orbit time.
"The danger with that thought process is that lower level team members were in effect making risk decisions for the Program, without necessarily having a Program wide viewpoint or understanding of the risk trades actually being made at a Program level."
Based on interviews, the board said "it was clear that several ground team members were concerned that if the assumed drink bag anomaly experienced at the end of EVA 22 were to be investigated further, it would likely lead to a long, intensive process that would interfere with necessary work needed to prepare for the upcoming EVA 23, and that this issue would likely not uncover anything significant enough to justify the resources which would have to be spent."
Along with this perceived schedule pressure, the board found that spacewalk engineers were lulled into a complacency by a misunderstanding of how water behaves in the weightlessness of space.
During ground testing, a water buildup in the cooling system would stall the pump fan and cause the system to shut down before any water could make its way into the helmet. As it turns out, water in a weightless environment can, in fact, make its way around the fan and into the vent system.
"When presented with the suggestion that the crew member's drink bag leaked out the large amount of water that was found in (Parmitano's) helmet after EVA 22, no one in the EVA community ... challenged this determination and investigated further," the board concluded. "Had that conclusion been challenged, the issue would likely have been discovered prior to EVA 23 and the mishap would have been avoided."
While the root cause of the contamination that triggered the leak in the first place is not yet known, the three U.S. spacesuits currently aboard the station have been thoroughly inspected and, in the case of Parmitano's suit, a replacement fan unit installed. Two contingency spacewalks were carried out in December to replace an ammonia coolant pump module after a failure that forced flight controllers to power-down non-critical systems. The spacewalks were successful and neither suit suffered any leaks.
But the mishap board made dozens of recommendations to prevent a repeat of the leak or any similar problems and Michael Suffredini, the space station program manager at the Johnson Space Center, said all of the high-priority directives would be addressed before the suits are used again for any non-contingency spacewalks, including efforts to improve communications between engineers and managers.
"It's about a balance," he said. "The report cautions us to make sure that we, in the process of emphasizing the significance of saving crew time for utilization, that we don't at a lower level accidentally cause people not to ask for crew time to do things they think are important," he said.
In the wake of the mishap board's conclusions, NASA managers have asked their teams if there is "anything we should be doing, if there's a risk they think we should be addressing and the answer came back 'no,'" Suffredini said. "And we spent quite a bit of time in all of our forums trying to reiterate with the teams that if there is ever a concern or a risk that they need crew time to work on, they should bring that forward to the program, that we welcome that sort of thing."
In a personal blog post before he left the station and returned to Earth, Parmitano succinctly summed up the challenge to engineers and astronauts.
"Space is a harsh, inhospitable frontier and we are explorers, not colonizers," he wrote. "The skills of our engineers and the technology surrounding us make things appear simple when they are not, and perhaps we forget this sometimes.
"Better not to forget."
CBS News
A fundamental misunderstanding of how water behaves in the cooling system of a spacesuit operating in weightlessness, combined with institutional complacency and a perceived pressure to avoid delays, resulted in a frightening, potentially fatal close call last summer when water flooded a spacewalker's helmet outside the International Space Station, NASA managers said Wednesday.
With his eyes and ears covered by a growing blob of water -- and faced with the counter-intuitive but very real prospect of drowning in the vacuum of space -- European Space Agency astronaut Luca Parmitano made it back to the safety of the station's airlock in the nick of time, assisted by fellow spacewalker Christopher Cassidy.
The July 16, 2013, incident was the most serious known incident ever encountered during a NASA spacewalk, and the space agency launched an extensive, wide-ranging mishap investigation to find out what went wrong and what steps need to be taken to prevent a recurrence.
 |
| Italian astronaut Luca Parmitano walks in space during a July 9, 2013, spacewalk. During a second spacewalk one week later, a water leak worsened and flooded his helmet in a potentially catastrophic malfunction. A mishap investigation board concluded the leak was caused by a poorly understood technical problem and institutional complacency that acted to prevent a more thorough discussion that might have caught the problem before the second spacewalk. (Credit: NASA TV) |
"While I am concerned about ensuring this particular incident does not happen again, I am especially concerned about cultural factors that may have contributed to the event," NASA Administrator Charles Bolden wrote in an agency-wide note to employees. "In our exuberance to get the job done, we may have allowed ourselves to accept the commonly accepted causes for small anomalies.
"We have a responsibility not to move on from any abnormal situation until we understand it fully or have suitable mitigations to prevent it happening again. Our work both in-house and with our industry and commercial partners should entail diligence in assessing risk and commitment to ensuring mission safety."
The mishap board concluded the leak was caused by a subtle water chemistry issue in the cooling system of Parmitano's spacesuit. Contamination built up and blocked a filter, allowing water to make its way into a line that feeds air to the helmet. The source of the contamination is not yet known.
A detailed review showed the leak actually began near the end of an otherwise-successful July 9 spacewalk. Parmitano and fellow spacewalker Christopher Cassidy initially suggested the water probably leaked from Parmitano's drink bag. Engineers on the ground came to the same conclusion and cleared the astronauts for a second spacewalk, EVA-23, on July 16.
Just 44 minutes into a planned six-and-a-half-hour excursion, Parmitano ran into a pair of problems. A carbon dioxide sensor in his suit suddenly stopped working and shortly after that, he noticed the back of his head felt unusually wet.
A CO2 sensor failure was not considered a major problem. In this case, flight controllers said they would monitor suit telemetry on the ground and keep tabs on how Parmitano was feeling. There were no indications of an actual CO2 buildup.
But as Parmitano pressed ahead with work to route cables on the outside of the International Space Station, he noticed a water buildup at the back of his head where a helmet air duct is located.
"I don't understand where it's coming from," he said.
"It has to be (your drink) bag," Cassidy replied. "Can you suck it dry?"
A few minutes later, Cassidy peered into his crewmate's helmet for a closeup look, saying "so that stuff on your forehead is not sweat?"
"No it's not."
The leak quickly grew and water began to creep around the side of Parmitano's head into the forward part of the helmet.
"I can still hear perfectly, but my head is really wet and I feel it increasing," Parmitano said. A few minute laster, he wondered again, "where's it coming from? It's too much."
"I don't know. It's a lot," Cassidy agreed.
"Now it's in my eyes," Parmitano said.
In the absence of gravity, water tends to pool in blobs and inside the cramped confines of a space helmet, a large amount of water could trigger uncontrolled coughing and, possibly, even drowning.
By that point, flight controllers assessing the problem at the Johnson Space Center in Houston knew the spacewalk could not continue. In fact, it needed to end as quickly as possible. The astronauts were told to return to the station's airlock as quickly as possible, a process that took about 40 minutes to complete. The excursion ended at 9:29 a.m., just an hour and a half after it began.
The astronauts later disassembled the suits and helped engineers on the ground carry out an extensive series of tests to pin down the source of the leak. They quickly concluded the drink bag was not the culprit. Why the ground team went along with that hypothesis in the first place was discussed at length by the mishap board.
Institutional complacency appeared to play a role in the incident, the board concluded, along with a perceived pressure to keep station science on track and not to raise questions or concerns that could trigger delays. The leak first appeared during the July 9 spacewalk, but the board concluded that spacewalk engineers jumped to conclusions when they blamed it on the drink bag.
"The ISS Program must place a strong emphasis on performing utilization with the ISS; it is in fact the very reason ISS exists," the board wrote. "However, the strong emphasis on utilization was leading team members to feel that requesting on-orbit time for anything non-science related was likely to be denied and therefore tended to assume their next course of action could not include on-orbit time.
"The danger with that thought process is that lower level team members were in effect making risk decisions for the Program, without necessarily having a Program wide viewpoint or understanding of the risk trades actually being made at a Program level."
Based on interviews, the board said "it was clear that several ground team members were concerned that if the assumed drink bag anomaly experienced at the end of EVA 22 were to be investigated further, it would likely lead to a long, intensive process that would interfere with necessary work needed to prepare for the upcoming EVA 23, and that this issue would likely not uncover anything significant enough to justify the resources which would have to be spent."
Along with this perceived schedule pressure, the board found that spacewalk engineers were lulled into a complacency by a misunderstanding of how water behaves in the weightlessness of space.
During ground testing, a water buildup in the cooling system would stall the pump fan and cause the system to shut down before any water could make its way into the helmet. As it turns out, water in a weightless environment can, in fact, make its way around the fan and into the vent system.
"When presented with the suggestion that the crew member's drink bag leaked out the large amount of water that was found in (Parmitano's) helmet after EVA 22, no one in the EVA community ... challenged this determination and investigated further," the board concluded. "Had that conclusion been challenged, the issue would likely have been discovered prior to EVA 23 and the mishap would have been avoided."
While the root cause of the contamination that triggered the leak in the first place is not yet known, the three U.S. spacesuits currently aboard the station have been thoroughly inspected and, in the case of Parmitano's suit, a replacement fan unit installed. Two contingency spacewalks were carried out in December to replace an ammonia coolant pump module after a failure that forced flight controllers to power-down non-critical systems. The spacewalks were successful and neither suit suffered any leaks.
But the mishap board made dozens of recommendations to prevent a repeat of the leak or any similar problems and Michael Suffredini, the space station program manager at the Johnson Space Center, said all of the high-priority directives would be addressed before the suits are used again for any non-contingency spacewalks, including efforts to improve communications between engineers and managers.
"It's about a balance," he said. "The report cautions us to make sure that we, in the process of emphasizing the significance of saving crew time for utilization, that we don't at a lower level accidentally cause people not to ask for crew time to do things they think are important," he said.
In the wake of the mishap board's conclusions, NASA managers have asked their teams if there is "anything we should be doing, if there's a risk they think we should be addressing and the answer came back 'no,'" Suffredini said. "And we spent quite a bit of time in all of our forums trying to reiterate with the teams that if there is ever a concern or a risk that they need crew time to work on, they should bring that forward to the program, that we welcome that sort of thing."
In a personal blog post before he left the station and returned to Earth, Parmitano succinctly summed up the challenge to engineers and astronauts.
"Space is a harsh, inhospitable frontier and we are explorers, not colonizers," he wrote. "The skills of our engineers and the technology surrounding us make things appear simple when they are not, and perhaps we forget this sometimes.
"Better not to forget."