By now, you’ve probably heard from Google just how safe its self-driving car is, and how it could prevent tons of accidents. But there’s a hidden danger you probably don’t know about.

Google has long claimed that its self-driving automobiles are safer than those driven by humans, and now it appears to have the data to back it up, with a new study — commissioned by Google — showing a huge 27 percent dip in accidents for self-driving cars compared to humans. Cars driven by humans had 4.2 crashes per million miles driven, whereas Google vehicles had a rate of just 3.2 crashes per million. And Google claims its self-driving cars are never at fault in an accident. All told, with 30,000 people dying in the United States from car accidents each year, that translates to thousands of lives saved every year if it holds up.

But there’s a catch — one big weakness to self-driving cars that could make them sometimes very dangerous. And that variable is us.

The problem is, at this point a self-driving car can’t be completely autonomous. At some point it has to hand over control to a human, and that is fraught with risks. At a moment’s notice, a human must quickly be able to orient himself or herself with what is going on around the car to assume control, and recent studies indicate that this doesn’t always go smoothly, meaning that a relaxing drive could turn into a nightmare if you don’t have your wits about you.

The reality is it’s tough to suddenly go from sitting in your seat reading a book or checking email and then suddenly taking control of an automobile at high speeds on the highway. And if the reaction speed isn’t good enough, it can result in disaster.

It’s a major technological question that developers are working on right now. It has so far not been possible to totally remove the human from the equation, and so scientists have a big task ahead of them as self-driving cars grow in prevalence: figure out how to keep the human from getting in the way.