Breakthrough: Robotic limbs moved by the mind

The following script is from "Breakthrough" which aired on Dec. 30, 2012 and rebroadcast on June 2, 2013. Scott Pelley is the correspondent. Denise Schrier Cetta, producer.

In a decade of war, more than 1,300 Americans have lost limbs on the battlefield. And that fact led the Department of Defense to start a crash program to help veterans and civilians by creating an artificial arm and hand that are amazingly human. But that's not the breakthrough. We don't use that word very often because it's overused. But when you see how they have connected this robotic limb to a human brain, you'll understand why we made an exception.

As we reported last December, to take this ultimate step they had to find a person willing to have brain surgery to explore new frontiers of what it is to be human. That person would have to be an explorer with desperate need, remarkable courage and maybe most of all, a mind that is game.

The person they chose is Jan Scheuermann, a Pittsburgh mother of two and writer, with a mind nimble enough to match wits on "The Wheel of Fortune" in 1995.

[Jan Scheuermann: I'm going to solve the puzzle."Too Cute For Words."]

When her mind triumphed her brain sent signals of delight to every muscle in her body. But a year after this moment, those brain signals were being cut off.

Jan Scheuermann: One day, I had trouble in the evening, I was making a lot of trips in and out of the car. It felt like my legs were dragging behind me.

Within two years a genetic disease called spinocerebellar degeneration broke the connection between brain and body. Now, at age 53, Jan Scheurmann can move only the muscles in her face and a few in her neck -- she's dependent on a caregiver for nearly all of her daily needs.

{Mother: 52 Across.]

And her mother to help her solve the puzzles she loves.

[Mom: Healed?

Jan Scheuermann: Oh, oh, you're good.

At the same time, Jan Scheuermann was putting her mind to a new life, a neuroscientist, just across town, at the University of Pittsburgh, was imagining how people like Jan might be restored.

Andy Schwartz, on the right, is working on an ambitious Defense Department project called "Revolutionizing Prosthetics."

Five years ago, we visited his lab and Schwartz showed us how he implanted tiny sensors like this one into the brains of monkeys and then wired them to a crude robotic arm.

Schwartz told us that, when the monkey thinks about moving his own arm, his brain cells, or neurons, fire off electrical signals. The sensor in his brain can pick up these signals and send them to the robot.

Scott Pelley: So he's operating the arm in three dimensions, up, down, forward and back?

Andy Schwartz: As well as the gripper.

Scott Pelley: What you're telling me is that the monkey is operating this arm with nothing but his thoughts?

Andy Schwartz: Absolutely.

Scott Pelley: What are the chances that a human being would be able to do this same thing?

Andy Schwartz: Oh, we think a human being could do much better.

That conversation was in 2008. And since then, the $150 million Revolutionizing Prosthetics program has reached farther than most thought possible.

Dr. Geoffrey Ling, a retired Army colonel and neurologist, is in charge. After seeing the wounded on several tours in Iraq and Afghanistan, he told his team that he wanted a breakthrough within five years.

Scott Pelley: Did any of them say, "Look, colonel, we're not sure we can do this."

Geoffrey Ling: Oh, absolutely.They, they thought we were crazy. But that's quite all right because I think it's in our insanity that things happen.

That madness led to genius in labs all across the country. At the Applied Physics Laboratory at Johns Hopkins University in Maryland, Michael McLoughlin led the multimillion dollar engineering of what has become the most sophisticated hand and arm ever developed. It's the same size and weight of an average man's arm and hand and everything is inside--including the computers and the batteries.

Scott Pelley: Is there anything that your natural arm and hand can do that the mechanical hand can't?

Michael McLoughlin: Well, I can do this.

Scott Pelley: Okay. There's that.

Michael McLoughlin: We can't do that. But other than that, virtually everything your natural hand can do, this prosthetic is able to do. Same strength too.

Scott Pelley: Same strength?

Michael McLoughlin: Same strength. So we can curl 45-50 pounds with the arm.

They've thought of a lot of ways to use it. When set on wheels it can bring a human touch where no human can go.

In this demonstration, we wore a visor that showed us the video feed from the robot. These gloves moved the robotic hands. And we practiced pulling a wire out of a bomb.

Scott Pelley: Come on, give me that pinch. Awesome.

But the Holy Grail in the project was finding a way to connect the robot directly to the brain.

Jan Scheuermann: Who wouldn't want to do this? When they told me-- I heard about the study, I said, "Oh, absolutely." I-- I couldn't not do this.

In 2012, Jan Scheuermann put herself on the line for a more sophisticated version of the surgery that they had done earlier in the monkeys.

Scott Pelley: There's a brain surgery involved. It's experimental. Why were you so excited about it?

Jan Scheuermann: I've always believed there's a purpose to my illness. I didn't think I would ever find out what it was in my lifetime. And here came this study where they needed me. You know, they couldn't just pick any Tom, Dick or Harry off the street. And in a few years, the quadriplegics and the amputees this is just going to help.The Department of Defense is funding some of this for the vets. To be of use to them and service to them, what an honor.

[Elizabeth Tyler-Kabara: What I'm going to do right now is I'm just going to make some marks here in your hair.]

The procedure was done by University of Pittsburgh neurosurgeon Elizabeth Tyler-Kabara, who showed us that the area that controls hand and arm movement is accessible right on the surface of the brain.

Scott Pelley: What are the dangers?

Elizabeth Tyler-Kabara: We worry about if we were to accidentally tear a blood vessel when we were putting them in that we could cause a blood clot that would collect on the surface of the brain. Probably the thing we worry about the most is the possibility of infection.

Scott Pelley: I mean, you do have a connection through the skull to the outside world?

Elizabeth Tyler-Kabara: Absolutely.

[Elizabeth Tyler-Kabara: May I have some irrigations]

During the six-hour surgery, two sensor arrays, each the size of a pea, were placed on the surface of Jan's brain. Then they were wired to two computer connections, called pedestals, the gateways to Jan's thoughts.

Scott Pelley: You know, people are going to look at those pedestals in your skull, and they're going to think, "That has to hurt." Is, is it painful? Has it been difficult in any way?

Jan Scheuermann: For a few hours after I woke up, I had the worst case of buyer's remorse. I was thinking, "Oh my God, I had brain surgery. Why didn't anyone stop me? Why didn't they say, "Jan, you're crazy." But as soon as the headache went away, that kind of talk went away too.

Five months after the surgery, we came back to see whether she would be able to control the robotic arm with nothing but her thoughts. They plugged her brain into the computer and this is what we saw.

Jan Scheuermann: I can move it up. And straight down. And left and right, and diagonally. I can close it. And open it. And I can go forward and back.

Scott Pelley: That is just the most astounding thing I've ever seen.

Scott Pelley: Can we shake hands?

Jan Scheuermann: Sure.

Scott Pelley: No, really?

Jan Scheuermann:: Yeah.

Scott Pelley: Like come right over here?

Jan Scheuermann: Yes, you come over there.

Scott Pelley: OK.

Jan Scheuermann: Let me grasp your hand there. There we go.

Scott Pelley: Oh my goodness.

Jan Scheuermann: Move it up and down a little.

Scott Pelley: Wow.

Jan Scheuermann: And I can do a fist bump if you'd like.

Scott Pelley: That's amazing.

Scott Pelley: What are you doing Jan? What's going on in your mind as you're moving this arm around? What are you thinking?

Jan Scheuermann: OK. The best way to explain it is, raise your arm. Now what did you think about when you did that?

Scott Pelley: Not much. I do it all the time.

Jan Scheuermann: Exactly. It's automatic.

Scott Pelley: Is that hard work? Are you having to concentrate?

Jan Scheuermann: It, no, it was hard work getting there. I struggled greatly to go up and down at the beginning. Now up and down is so easy I don't even think about it. Side to side, don't even think about it.

Scott Pelley: Just like your arms used to?

Jan Scheuermann: Yes.

We asked Dr. Ling, the program manager, where all of this is headed.

Geoffrey Ling: I'm old enough to have watched Neil Armstrong take that step on the moon. And, and to watch Jan do that, I had the same tingles. Because I realized that we have now stepped over a great threshold into what is possible. And very importantly, what patients can now expect in terms of restoration. This is a very important part. Not rehab, but restoration of function.

Scott Pelley: I wonder what your experience with Jan has taught you about the brain - and the brain's ability to adapt to new circumstances.

Geoffrey Ling: I think it's taught me something really fundamental, and that is we are tool users. And our arms and legs are just tools for our brain. And so when we give another tool, in Jan's case, a robot arm, she will adapt to that tool to do the things that she wants to do.

Of course, many who could use a robot arm are not paralyzed like Jan. They're amputees. And for them, the project has found a way to connect the arm without brain surgery.

58-year-old Johnny Matheny lost his arm to cancer. Dr. Albert Chi, from Johns Hopkins Hospital, found nerves that used to go to Johnny's hand and moved them to muscles in his remaining limb.

[Albert Chi: Now elbow extension.]

Sensors on his skin pick up the brain's signals from the nerves and use those signals to control the robotic arm.

[Johnny Matheny: Come here, I want to see you.]

Scott Pelley: So even though the limb is missing, the brain still sends the signals as if the limb was still there?

Albert Chi: Correct.

Scott Pelley: Johnny, it feels in your mind like your hand is there again?

Albert Chi: Yes.

Scott Pelley: As if your arm had never been lost?

Johnny Matheny: Correct.

Unlike Jan, the connection for Johnny runs both ways. Sensors in the fingers send signals back so he can feel what he's touching.

Scott Pelley: OK, I'm holding the object and you can close on it.

To see how well, we put him to the test.

Scott Pelley: Hard or soft?

Johnny Matheny: Soft.

Scott Pelley: Correct. Very good. Now let's try again. I'm holding the object. Hard or soft?

Johnny Matheny: Soft.

Scott Pelley: Yep. Quite right. All right.

He got it right every time.

Scott Pelley: Hard or soft?

Johnny Matheny: Hard.

Scott Pelley: Amazing.

The next person to have Jan's surgery will have additional sensors placed in the brain to receive the sensation of touch. Andy Schwartz believes that will help with some of the things that Jan has trouble with. For example, sometimes when she looks right at an object she can't grab it.

Andy Schwartz: OK. I'm going to take the cone away. Just go ahead and close it.

Jan Scheuermann: Oh sure, no problem.

Andy Schwartz: So as soon as I take the cone away there is no problem. But as soon as I put the cone there, she can't do it.

Why, is still a mystery.

The progress is coming rapidly. They are working on a wireless version of the implant to eliminate the connection in the skull. And Dr. Geoffrey Ling told us the lab experiments will one day enter the real world.

Geoffrey Ling: And we're going to not stop at just arms and hands. I think that it's going to open the way for things like sight and sound. And my dream, I dream that we'll be able to take this into all sorts of patients. Patients with stroke, patients with cerebral palsy, and the elderly.

Jan Scheuermann: I think when other quadriplegics see what I'm doing with the arm, they're all gonna say, "Oh wow! I wish I could do that!"

[Geoffrey Ling with Jan: Now this is the way I like to eat cookies. Awesome. Thank you so much. ]

Jan Scheuermann: And I just feel very honored to be the one who gets to do it.