New internal Uber documents leaked to Recode detail the company’s progress toward realizing its dream of a fleet of vehicles entirely devoid of pesky human drivers.
As those files reveal, Uber’s month-over-month metrics aren’t exactly a steady line of progress, more a jerky sort of stumbling toward its goal of self-driving reliability. And as Uber’s court battle with Google over autonomous car tech begins, that visual is a particularly apt metaphor.
Uber’s self driving fleet, spread across Pennsylvania, California and Arizona, is driving more miles than ever, but its vehicles aren’t improving in a steady way on measurements of rider experience.
Uber breaks this variable down into a few different data streams: how many miles a car makes it before a human takes over for any reason which it calls “miles per intervention,” how many miles a car goes before a “critical” driver takeover (to avoid harm or damage) and how many miles a car goes before a “bad experience,” a measure of overall ride smoothness that is less focused on safety.
By the miles per intervention measure, Uber’s fleet isn’t doing so hot. In January, an Uber autonomous vehicle could drive .9 miles before a driver takeover. By February, that number had inched up to one full mile before dropping down again to .71 miles. As of last week it was .8 miles.
When it comes to measures of critical interventions — the scary, accident-avoiding ones — Uber’s metrics are trending upward, albeit erratically. At the start of February, an autonomous vehicle could make it 125 miles without a critical intervention, but the following week that number dipped down to 50 miles. By the third week in February it shot back up to 160 miles before dipping to 115 again the following week. At the last measure, taken the week of March 8, it was up to 196 miles.
By measures of “bad experiences” like hard stops and jerky driving, the fleet is getting worse. In mid-January, Uber self-driving cars averaged 4.5 miles before a bad experience, but by the next month that had dropped down to 2 miles, where the number remained into the first week of March.
Recode’s numbers paint a rough outline of Uber’s autonomous vehicle performance, but it’s worth remembering that the cars are still figuring out routes and learning as they go. Still, as the numbers illustrate, Uber’s fleet is demonstrating some fairly wild swings on measures of safety and reliability. The numbers may be improving in aggregate, but their erratic movement doesn’t exactly inspire confidence when it comes to taking your hands off the wheel.
