Hands off the Wheel

The following is a script from "Hands Off the Wheel" which aired on Oct. 4, 2015, and was rebroadcast on Dec. 6, 2015. Bill Whitaker is the correspondent. Marc Lieberman, producer.

Car accidents cost us much more than time and money. They also take a staggering number of lives. Every year on American roads, nearly 33,000 people die, almost all because of driver error. That's the equivalent of a 747 full of passengers crashing once a week for a year. Self-driving cars could save more than two-thirds of those lives. That's what the nation's top auto regulator told us.

It's no wonder the biggest names in the auto industry and high-tech are racing to develop driverless cars powered by a form of artificial intelligence. Six years ago, Google rolled out a prototype that jumpstarted the competition. Today, Apple and Uber are experimenting too. We wanted to see how far the technology has come. So, as we first reported earlier this year, we hit the road in Silicon Valley, the new Detroit for self-driving cars.

Bill Whitaker: What do you have to do to make the car take over?

Ralf Herrtwich: I just pull this lever. And now--

Bill Whitaker: System is active?

Ralf Herrtwich: It goes.

Computer scientist Ralf Herrtwich runs autonomous vehicle research for Mercedes-Benz. He punched in a route and took us for a 20-mile drive on city streets and highways in this S500, the company's most advanced self-driving prototype.

Bill Whitaker: So this is like no hands, no feet, car is in charge?

Ralf Herrtwich: Yeah, the car is in charge.

Right from the start, the car astonished us. As we approached our first intersection, it slowed down and steered itself into the left turn lane.

It's a German car so naturally it has a German accent. That was the voice of Herrtwich's secretary.

Bill Whitaker: So it just took off by itself when the light turned green and now it's making this left turn by itself, with other traffic around. This is absolutely amazing.

Just two minutes into the ride, we entered a freeway onramp. If you think a normal merge is nerve wracking, try it with a driver who's talking with his hands.

Bill Whitaker: I must admit, I find it a little disconcerting that we are driving toward the freeway, and you don't have your hands on the wheel.

Ralf Herrtwich: Shall I put them back on? Would that make you feel more comfortable?

Bill Whitaker: No, no, no.

"Some people have remarked that the car itself, in some cases, drives a bit like an old lady. That's...that's fine with us, for the time being."

Herrtwich gave us a rare opportunity to go on an actual test run near Mercedes' Silicon Valley lab. Almost every major automaker is working on the technology here. Nissan has teamed up with NASA. Auto-parts maker Delphi put its system in this Audi. It was the first to drive itself across the country. Back at that merge, don't hold your breath for the car to step on it. This S500 won't break the speed limit.

Bill Whitaker: Are you gonna have little old ladies driving up behind you, beeping the horn to get going, get moving?

Ralf Herrtwich: Some people have remarked that the car itself, in some cases, drives a bit like an old lady. That's...that's fine with us, for the time being.

Especially since the car has driven about 20,000 miles without an accident. Mercedes made its name selling the passion for driving on the open road. Now it sees a future in the growing desire to be driven through traffic-jammed streets.

Bill Whitaker: What's fueling this?

Ralf Herrtwich: People are increasingly asking for this. People probably have become used to live more with computers and interact with computers. And they feel more comfortable doing this. And so, all of a sudden we see this interest. And hey, there are certain situations where I don't want to drive. Can your car do it for me?

First you're amazed, then you begin to relax. Surprisingly, it took less than 10 minutes to feel comfortable with the car in control.

Bill Whitaker: This is amazing.

But don't get too comfortable.

Ralf Herrtwich: This is not good.

Those beeps...that's not a sound you want to hear. It means the car senses trouble and needs a helping human hand.

Ralf Herrtwich: Now the vehicle asks me to take over.

At this intersection, that silver car got too close.

Ralf Herrtwich: This is-- for example, I-- rather took over. It would've managed. But I, really it was-- this was too close for us.

Bill Whitaker: That guy was getting into our lane there?

Ralf Herrtwich: Yeah.

It only happened a few times while we were driving around. Herrtwich says teaching the car to handle encounters like that silver car -- on chaotic city streets with impulsive human drivers -- will keep his engineers busy for the next decade.

Bill Whitaker: I'm not an engineer. But how do you figure things like that out?

Ralf Herrtwich: The important thing about an autonomous vehicle is it has to have a very good sense of its environment. A vehicle cannot react to something it does not see. So we have to be very careful that we see everything that happens around us.

The car sees with an array of cameras and radar sensors designed into the body, constantly scanning up to 600 feet in all directions.

Ralf Herrtwich: We can actually detect more quickly that something is happening-- that may cause an accident than the human driver can.

Bill Whitaker: So these cars would actually be safer, you're saying, than a human driver?

Ralf Herrtwich: That's what we aim for.

"The important thing about an autonomous vehicle is it has to have a very good sense of its environment. A vehicle cannot react to something it does not see. So we have to be very careful that we see everything that happens around us."

That's what Google is driving for too. Its autonomous cars rely on roof-mounted laser sensors to see the road. In the last six years, its fleet has driven more than a million miles.

Chris Urmson: We're getting to a place where we're comparable to human driving today.

Robotics scientist Chris Urmson is the director of Google's self-driving car project. He invited us inside his Silicon Valley garage where the autonomous future is taking shape.

Bill Whitaker: Google's a tech company, not a carmaker?

Chris Urmson: Absolutely. But the heart of what makes the technology work is the algorithms and the software. And that's one of the things that we are really quite good at.

Bill Whitaker: There are so many variables, so many different scenarios. How is it possible to put all of that knowledge into a car?

Chris Urmson: And that's really the trick, right? And that's what makes this hard. You can't just kind of go through and enumerate, you know, the thousand different scenarios it might encounter, 'cause it's not 1,000. There's an infinite number of them, right? And so the trick is to develop these algorithms that can generalize.

By generalize, he means "think" and this is how it works. The algorithms are trained to recognize other cars, pedestrians, cyclists, and animals from their movements, size, and shape. Each car's daily driving experience is analyzed, uploaded and shared. The cars can then make predictions and choices based on the collective knowledge of the fleet. Look in the lower left corner as one of Urmson's cars encounters a pickup truck that stops to parallel park.

Bill Whitaker: Now, how does the computer know that it's someone intending to back into a parking space, and not someone who's just stopped in the street?

Chris Urmson: Our cars have seen thousands and thousands of vehicles. And they get a feeling, you know, they get a feeling really for what the behavior of those vehicles are going to be.

Bill Whitaker: Really?

Chris Urmson: So its seen lots of cars backing up and so it understands if there's a space here, and a car stopped just in front of it, that means it's gonna probably back into that spot.

Bill Whitaker: My smartphone has computer glitches. My computer has glitches. How do you get people to trust that this computer-on-wheels is not going to have a glitch?

Chris Urmson: We're all used to our bits of home computing doing funny things, right? And what you have to remember is they're engineered and designed very differently. The way we develop the software, the way we develop the hardware, you know, the way we think about redundancy, the way we think about the situations it has to deal with on the road, it's completely different.

Right now, the technology can't handle snow. Google's cars can't operate in heavy rain. The Mercedes S500 can't decipher hand gestures from traffic cops or pedestrians. Four million miles of roads in the U.S. must be mapped in ultra high-definition detail. The automakers call these solvable problems. In the meantime, the car industry plans to automate the driving experience feature-by-feature, what some are calling revolution-by-evolution.

The revolution is already being televised in ads.

[Infiniti ad: Backup collision intervention which can brake, even before you do.]

In showrooms today, you can buy features to automatically keep you in your lane, help you park, drive you in stop-and-go traffic and recently this: hands-free highway driving. Tesla made it available in October. GM plans to offer it in a 2017 Cadillac.

We are at probably the largest transformative moment in the history of the automobile.

Mark Rosekind is head of the National Highway Traffic Safety Administration. He is optimistic but also realistic about this new technology.

Mark Rosekind: This is really different than just thinking about the engine parts and the tires. Now we're talking about cars are computers. So issues related to cybersecurity and privacy are just as big an issue as the defect in the manufacturing process.

Bill Whitaker: Someone can hack your computer and steal your money. But someone can hack your car and you can die?

Mark Rosekind: People have to trust these vehicles. If they read or suspect in any way that they literally could be one virus away from a crash occurring, they're not going to get in that car. They're not going to buy it, they're not going to let it drive them. That whole future evaporates.

Rosekind also worries about a future in which drivers place too much trust in the cars.

Mark Rosekind: Think about how some of this is being sold. "Oh, you can take a nap. You can read the paper." What would you do if you had to take over in a certain emergency situation? Nobody has that future totally nailed yet.

Mercedes and other major carmakers say humans will always have a role in driving. But Chris Urmson of Google says it's dangerous to require humans to snap to attention and take control at a moment's notice. So the company stopped developing cars that put humans on call. Now, it's testing 30 fully autonomous electric prototypes custom built for the job.

Bill Whitaker: So I would punch in where I wanted to go and it would just take off and go there?

Chris Urmson: And it'd take off, you press the little go button under here. Pull away from the curb, take you where you wanted.

For safety, the cars max out at 25 miles per hour. They don't need steering wheels or pedals, but they have them to comply with current California law.

[Jaime Waydo: The goal of this is to improve the remote assistance link?]

Jaime Waydo oversees the engineering. She used to work at NASA on autonomous vehicles of a different sort: the Mars rovers.

Jaime Waydo: Doing self-driving cars here on Earth is actually more challenging in a lot of ways.

Bill Whitaker: More difficult than driving across the surface of Mars?

Jaime Waydo: I think so. Humans are so unpredictable. And so having to try to have a car who can out predict an unpredictable human is amazing and really, really hard.

Google's cars have been in 10 minor accidents in self-driving mode -- all, the company says, the fault of humans driving in the other cars. Google and Mercedes told us, if their technology is at fault once it becomes commercially available, they'll accept responsibility and liability. But all involved expect fewer crashes as the technology evolves. For now, it's accelerating to the near future and beyond. This is Mercedes' vision for the year 2030, the F015.

Peter Lehmann: So we have an app.

You can summon it with your phone.

Peter Lehmann: The car will start and come to you.

German engineer Peter Lehmann took us for a test drive at an old naval base on San Francisco bay. The car's radical design was shaped by expectations of life in the future.

Bill Whitaker: You turn your back to the steering wheel.

Mercedes is planning for overcrowded cities, perpetual gridlock, and an autonomous car to drive the stress away.

Peter Lehmann: Now you can relax. Or you can-- look a movie. So you have really gained time.

Bill Whitaker: I feel like I'm driving into the future right now.

Peter Lehmann: Ah ha, yes.

A future Google's Chris Urmson says is coming and coming fast.

Bill Whitaker: So how long before that day?

Chris Urmson: So the way I talk about this is, I have two children, 11 and 9-year-old. And the 11-year-old is going to be able to get a driver's license in about four and a half years. And my mission is to make sure that doesn't happen.

Bill Whitaker: You want him to have a driverless car?

Chris Urmson: I want him to have a driverless car.