Coolant problem triggers powerdowns aboard ISS (UPDATED)
07/31/2010 11:50 PM Filed in: International Space Station | Space News
Editor's note...
Trouble with one of the International Space Station's external coolant loops, used to dissipate the heat generated by the lab's electronics systems, triggered an extensive powerdown late Saturday. NASA managers met Sunday and gave preliminary approval to a difficult two-spacewalk repair job, starting as early as Thursday, to restore the critical system to normal operation.
It is not yet clear what went wrong, but the ammonia pump module that is part of coolant loop A, mounted on the right side of the station's main power truss, failed around 8 p.m. EDT Saturday. A problem somewhere in the system caused a circuit breaker to trip, setting off multiple alarms and waking the crew.
A graphic showing the location of the S1 ammonia pump module and its replacement.
(Credit: NASA)
With half the station's cooling gone, flight controllers were forced to shut down two of the station's four U.S. control moment gyroscopes, used to help maintain the lab's orientation in space, one communications channel, several solar power current converters and a variety of computer control boxes known as multiplexer-demultiplexers.
Tracy Caldwell Dyson and Douglas Wheelock assisted with the powerdowns and hooked up jumper cables between the Russian Zarya module and the U.S. segment of the station to prevent additional cooling problems.
"It seems like we're in a sim right now," Caldwell Dyson joked with ground controllers shortly after 1 a.m. Sunday.
The space station remained in a safe configuration throughout, officials said, with critical life support systems, computers and communications gear operating with coolant loop B. The six-member crew, they said, was never in any danger.
"We're going to be working hard overnight to figure out what's going on," astronaut James M. Kelly radioed from mission control shortly before the astronauts went back to bed. "By tomorrow, a little bit later on, hopefully we'll be able to send you up a little bit better idea of where we stand on everything."
Shortly before 6 a.m., flight controllers attempted to restart the stalled pump, resetting the circuit breaker that opened late Saturday. Once again, the crew was awakened by alarms.
"Sorry about that. We thought we had (the alarms) taken care of," Jay Marschke called from Houston, apologizing for another unplanned wakeup call. "We were trying to turn on the loop alpha pump to see if there was any life in it. Unfortunately, we got a recurrence of the problem and it shut down. No other actions at this time for you guys, everything else is nominal, we're still in our jumper contingency configuration."
Later in the morning, engineers restarted one of the two powered-down control moment gyroscopes and while the main bus switching units that direct power to various subsystems ran slightly hotter than normal, engineers said the lab was stable.
"Loop bravo is rejecting heat just fine," Marschke reported in an evening update for the crew. "The teams here are monitoring the temperatures on MBSUs 1 and 4, they're all currently within limits and seem to be holding steady. The biggest impact at this point is (loss of power to a component in the Unity module that) caused us to power off half the JEM (Japanese Experiment Module) and Columbus (laboratory module) loads. ... The team's done a great job of getting back full redundancy on tier 1 and tier 2 MDMs. Thanks again for installing that jumper."
"We know you're doing your job, but we know you're also thinking about us and our comfort and our awareness and we can't stress enough how much we appreciate the response we've gotten up here," Caldwell Dyson replied.
The space station features two independent coolant loops that use ammonia circulating through huge radiators to dissipate the heat generated by the station's electronic systems. Each loop is fed by a large tank of ammonia that includes an internal bellows pressurized by nitrogen. That pressurization system allows the loops to handle the periodic expansion and contraction of the ammonia coolant due to temperature changes in orbit.
"Losing one of those loops is very significant," space station flight director Brian Smith said before a shuttle flight last year. "We'd lose cooling capability to half of the electronics on the U.S., European and Japanese part of the space station."
Spare coolant system components, including four pump modules, are mounted on external stowage platforms.
An ammonia coolant pump module, with insulation removed (right) and in place (left).
(Photo: NASA)
Wheelock and Caldwell Dyson already were scheduled to perform a spacewalk Thursday to mount a robot arm attachment fixture to Zarya and to prepare the station for attachment of a new U.S. storage module in November.
Those tasks will now be deferred. Instead, NASA managers gave preliminary approval to a plan that calls for Wheelock and Caldwell Dyson to remove the faulty pump module during a spacewalk no earlier than Thursday. The old unit will be temporarily attached to a mounting fixture and the spare module will be moved from External Storage Platform No. 2 to the S1 truss segment for installation.
The 780-pound pump module, built by Boeing, will require numerous electrical and ammonia coolant line connections. Flight planners hope to get the new pump mechanically attached by the end of the first spacewalk. A second outing likely will be required two to three days later to make all the connections.
But mission managers caution that the preliminary plan may change, depending on the results of training runs and analysis at the Johnson Space Center. Caldwell Dyson and Wheelock rehearsed a pump module replacement in 2009 and engineers prepared updated procedures for an on-board training session Monday.
The ammonia tank on the left side of the station's power truss, part of coolant loop B, was replaced in August 2009. During a flight by the shuttle Discovery last April, astronauts replaced the tank on the right side in coolant loop A. Other than alignment problems and trouble getting a few bolts to engage, the tank swap went well and there were no indications of any major issues. But flight controllers were unable to re-open the main nitrogen valve needed to re-pressurize coolant loop A.
After Discovery undocked, engineers resolved the problem, cooling the valve and using back pressure to get it open. There have been no other obvious signs of trouble with either coolant loop since then.
- Updated at 01:30 a.m., 8/1/10, with additional details
- Updated at 07:25 a.m. with failed attempt to restart pump
- Updated at 01:55 p.m. with tentative plans for repair spacewalk
- Updated at 03:25 p.m. with evening status
- Updated at 10:25 p.m.; four pump modules, not two, aboard station
Trouble with one of the International Space Station's external coolant loops, used to dissipate the heat generated by the lab's electronics systems, triggered an extensive powerdown late Saturday. NASA managers met Sunday and gave preliminary approval to a difficult two-spacewalk repair job, starting as early as Thursday, to restore the critical system to normal operation.
It is not yet clear what went wrong, but the ammonia pump module that is part of coolant loop A, mounted on the right side of the station's main power truss, failed around 8 p.m. EDT Saturday. A problem somewhere in the system caused a circuit breaker to trip, setting off multiple alarms and waking the crew.
(Credit: NASA)
With half the station's cooling gone, flight controllers were forced to shut down two of the station's four U.S. control moment gyroscopes, used to help maintain the lab's orientation in space, one communications channel, several solar power current converters and a variety of computer control boxes known as multiplexer-demultiplexers.
Tracy Caldwell Dyson and Douglas Wheelock assisted with the powerdowns and hooked up jumper cables between the Russian Zarya module and the U.S. segment of the station to prevent additional cooling problems.
"It seems like we're in a sim right now," Caldwell Dyson joked with ground controllers shortly after 1 a.m. Sunday.
The space station remained in a safe configuration throughout, officials said, with critical life support systems, computers and communications gear operating with coolant loop B. The six-member crew, they said, was never in any danger.
"We're going to be working hard overnight to figure out what's going on," astronaut James M. Kelly radioed from mission control shortly before the astronauts went back to bed. "By tomorrow, a little bit later on, hopefully we'll be able to send you up a little bit better idea of where we stand on everything."
Shortly before 6 a.m., flight controllers attempted to restart the stalled pump, resetting the circuit breaker that opened late Saturday. Once again, the crew was awakened by alarms.
"Sorry about that. We thought we had (the alarms) taken care of," Jay Marschke called from Houston, apologizing for another unplanned wakeup call. "We were trying to turn on the loop alpha pump to see if there was any life in it. Unfortunately, we got a recurrence of the problem and it shut down. No other actions at this time for you guys, everything else is nominal, we're still in our jumper contingency configuration."
Later in the morning, engineers restarted one of the two powered-down control moment gyroscopes and while the main bus switching units that direct power to various subsystems ran slightly hotter than normal, engineers said the lab was stable.
"Loop bravo is rejecting heat just fine," Marschke reported in an evening update for the crew. "The teams here are monitoring the temperatures on MBSUs 1 and 4, they're all currently within limits and seem to be holding steady. The biggest impact at this point is (loss of power to a component in the Unity module that) caused us to power off half the JEM (Japanese Experiment Module) and Columbus (laboratory module) loads. ... The team's done a great job of getting back full redundancy on tier 1 and tier 2 MDMs. Thanks again for installing that jumper."
"We know you're doing your job, but we know you're also thinking about us and our comfort and our awareness and we can't stress enough how much we appreciate the response we've gotten up here," Caldwell Dyson replied.
The space station features two independent coolant loops that use ammonia circulating through huge radiators to dissipate the heat generated by the station's electronic systems. Each loop is fed by a large tank of ammonia that includes an internal bellows pressurized by nitrogen. That pressurization system allows the loops to handle the periodic expansion and contraction of the ammonia coolant due to temperature changes in orbit.
"Losing one of those loops is very significant," space station flight director Brian Smith said before a shuttle flight last year. "We'd lose cooling capability to half of the electronics on the U.S., European and Japanese part of the space station."
Spare coolant system components, including four pump modules, are mounted on external stowage platforms.
(Photo: NASA)
Wheelock and Caldwell Dyson already were scheduled to perform a spacewalk Thursday to mount a robot arm attachment fixture to Zarya and to prepare the station for attachment of a new U.S. storage module in November.
Those tasks will now be deferred. Instead, NASA managers gave preliminary approval to a plan that calls for Wheelock and Caldwell Dyson to remove the faulty pump module during a spacewalk no earlier than Thursday. The old unit will be temporarily attached to a mounting fixture and the spare module will be moved from External Storage Platform No. 2 to the S1 truss segment for installation.
The 780-pound pump module, built by Boeing, will require numerous electrical and ammonia coolant line connections. Flight planners hope to get the new pump mechanically attached by the end of the first spacewalk. A second outing likely will be required two to three days later to make all the connections.
But mission managers caution that the preliminary plan may change, depending on the results of training runs and analysis at the Johnson Space Center. Caldwell Dyson and Wheelock rehearsed a pump module replacement in 2009 and engineers prepared updated procedures for an on-board training session Monday.
The ammonia tank on the left side of the station's power truss, part of coolant loop B, was replaced in August 2009. During a flight by the shuttle Discovery last April, astronauts replaced the tank on the right side in coolant loop A. Other than alignment problems and trouble getting a few bolts to engage, the tank swap went well and there were no indications of any major issues. But flight controllers were unable to re-open the main nitrogen valve needed to re-pressurize coolant loop A.
After Discovery undocked, engineers resolved the problem, cooling the valve and using back pressure to get it open. There have been no other obvious signs of trouble with either coolant loop since then.