Why Has Waymo Taken So Long to Commercialize Autonomous Taxis?

In November 2017, Waymo announced it was testing autonomous cars in Phoenix, without drivers behind the steering wheels but with safety engineers in the back seats. Next, the company plans to commercialize an autonomous taxi service in Chandler, Arizona.

According to data from the California Department of Motor Vehicles,Waymo’s [GOOG] autonomous cars are improving rapidly enough to support a commercial launch within the next two years. The same data, however, also suggest that Waymo will face challenges, perhaps explaining its decision to keep the engineers in the car. Nonetheless, competition is brewing, reason enough to launch early to achieve first mover advantage in the autonomous car market, which ARK estimates will generate trillions of dollars in revenue by the late 2020s.

One challenge facing autonomous cars are failures. ARK classifies autonomous car failures in two categories:

(1) Expected Failures (EFs), or incidents during which the autonomous system is confused and knows it’s confused, signaling to a remote operator for help; and

(2) Unexpected Failures (UFs), or incidents during which the autonomous system is confused but does not know it’s confused, continuing to drive and potentially crashing in the absence of human intervention.

ARK envisions that while a remote network of third party operators will be able to direct autonomous taxis in the event of Expected Failures, it will face significant challenges in anticipating and responding to Unexpected Failures. To prevent or address UFs, remote operators would have to monitor each autonomous taxi continuously which, even if possible, would add substantially to the per-mile cost of an autonomous service. Network latency and other technical issues would prevent even the most diligent operators from intervening in time to prevent most if not all UFs. Consequently, Waymo and other service providers must drive down the UF rate substantially before autonomous taxi networks commercialize.

What is An Acceptable Failure Rate for Autonomous Cars?

To estimate the rate at which passengers will tolerate autonomous taxi failures, we analyzed the manually driven car statistics to set the hurdle. On average, human driven cars break down roughly once every 50,000 miles and crash once every 240,000 miles,[1]  thus offering perspective on acceptable tolerance rates for autonomous vehicle EFs and UFs, respectively, as shown below.

To benchmark the progress of autonomous technology, ARK examined Waymo’s California fleet which, as measured by failure rates,[2] is the most advanced autonomous system in the US, if not the world.

Today, Waymo’s autonomous test cars can drive for roughly 30,000 miles on average with no need for human intervention. If Waymo were to commercialize its autonomous efforts today, its riders would probably experience a remote operator intervention every 3 years. For perspective, every 30,000 miles, these disruptions would occur roughly twice as often as personal car breakdowns. Also note that while a breakdown today may involve hours of waiting for AAA, an autonomous taxi could fail by pulling itself over and a remote operator could take a few minutes or less to assist an autonomous taxi in need. At best, the passenger may not even be aware of a disruption if the handoff of driving responsibility from the car’s computer to the operation center is quick and seamless enough.

If progress were to continue at the rate of ARK’s forecast, and even factoring in a slight delay to account for the increasing difficulty of unsolved issues, Waymo’s cars should be ready for commercial deployment by 2019. On an annual average basis, Waymo tripled the number of miles between interventions from 2015 to 2016, but in 2017 saw only a 20% improvement in intervention rate, along with some quarterly turbulence as shown below. However, this average may be deceiving, and a look at the underlying types of disengagements may shed more light on Waymo’s progress.

Interpreting Waymo’s Progress

According to data submitted to the California DMV, the improvement rate in UFs—the errors most difficult to manage—had stagnated from the end of 2015 through the first half of 2017, signaling perhaps that Waymo’s car could be further from commercialization than the disengagement rate’s longer-term trajectory had suggested in mid 2016. Supporting this hypothesis, its cars seemed to have difficulty making left turns. One possible explanation is that Waymo has chosen not to vertically-integrate, outsourcing vehicle production to partners like Fiat Chrysler and integrating its sensor suite into a product manufactured outside of its control. In contrast, Tesla’s [TSLA] and Cruise Automation’s [GM] manufacturing operations are vertically-integrated, which could become an important source of competitive advantage.

However, our research shows that in 2017, Waymo’s UF rate improved from roughly 1 every 5,000 miles to 1 in 30,000. Interestingly, over the same time period, EFs became more frequent. If Waymo is able to continue to decrease surprises, or UFs, an increase in EFs may be justified as these could easily be managed by a remote operator network of drivers.

While Waymo may be on track to surpass the 50,000 mile UF rate milestone in 2018, it probably will miss the more critical 240,000 mile UF parity car crash rates, as shown in the two charts below. In other words, based on recent evidence alone, critics could conclude that autonomous taxis will not be ready for prime in the next two decades.

We are skeptical of that negative conclusion for a number of reasons. Today, Waymo probably is trying to maximize its failure rate to identify faults and root them out. Some stretches of road are trickier and some intersections more difficult to navigate than others. In Los Angeles, for example, roughly a quarter of pedestrian collisions take place at only 1% of its intersections. By testing vehicles in the most challenging venues, Waymo probably is experiencing failure rates much higher than would otherwise be the case. Moreover, only in California does the Department of Motor Vehicles require the reporting of interventions. Waymo is testing cars in Washington, Arizona, and Texas, where the geographies and test results could be much different. Finally, Waymo is likely to launch its commercial service in safer geographies and terrains, gathering lots of data on miles traveled in order to train and improve the service. Chandler, Arizona, a suburb of Phoenix, seems ideal for initial rollout, with good weather, simple roads, and limited government oversight.

Without a doubt, the stakes are high. Tesla, Baidu [BIDU], and GM all have plans to launch autonomous services within the next two years. The company first to market could accrue an insurmountable data advantage which, given a market opportunity worth trillions of dollars, makes the risks worth taking sooner rather than later.

Cover Image: Uploaded by Grendelkhan is licensed under CC BY-SA 4.0