Is Waymo ready for winter?

During a recent comprehensive meeting, Waymo’s chief winter weather expert I put it to all the company’s engineers and product managers: If Waymo wants to expand into new cities and new markets, its robotaxis must be able to handle snow confidently and safely.

For its first few years of operation, Waymo strategically stuck to cities with warmer, drier climates — places like Phoenix, Los Angeles, Atlanta, and Austin. But as it looks to a host of East Coast cities, including Boston, New York and Washington, D.C., in the next phase of its expansion, its ability to handle more bad weather will become a critical test.

“Haha, good question,” said Robert Chen, the company’s weather product lead, when I asked him when he hoped to validate a Waymo driver for winter operation. “This winter is going to be a really important season for us. I think that’s all I can say at this point.”

There is an understanding that the inability to handle winter roads could hamper the usefulness of Waymo’s robotaxi service, which could limit its growth in new cities. Other human-powered transportation services drive in all weather conditions. If Waymo wants to compete, it can’t just be a fair-weather robo-taxi company.

“I think here at Waymo we really aspire to build this product and service that people can rely on and use, not just eight, nine, 10 months out of the year, but something they can really rely on and use whenever they want,” Chen said.

In some ways, autonomous vehicles are very similar to human drivers: they work best with good visibility, dry roads, and sensors clear of obstacles. When the roads become icy and snow accumulates, things get complicated. Waymo has faced a variety of challenges, from flash floods to dust storms in Phoenix known as “blows,” but snow presents a unique challenge, he said.

Phil Koopman, an expert in self-driving vehicle technology from Carnegie Mellon University, said human drivers can make assumptions about road markings and signals that may be off the mark for more trained robots. Snow can obstruct road markings and street signs, making it more difficult for a vehicle’s perception system to respond safely and accurately.

“You may only see a third of a stop sign, but you know it’s a stop sign,” Koopman said. “Machine learning could have a problem with that if it’s not trained on partially obstructed stop signs.”

A system like Waymo’s, with a whole host of redundant hardware like lidar, radar and cameras, eventually has to be up to the task, Koopman added. Camera-only systems, like Tesla’s, may have more problems.

“It would be easier for a multi-sensor platform because the cameras would have a lot of problems in blowing snow,” he said. “But certainly radar will really help if there is snow.”

This is not only a technical challenge, but also a data challenge as well. Snowy conditions are rare in Waymo’s data set, making relevant data very sparse — often less than 5 percent or even a fraction of a percent of the total driving data for rarer conditions, Chen said.

This scarcity has prompted Waymo to use innovative technologies, including advanced artificial intelligence methods, to leverage and analyze data for development and validation, which Chen said is helping the company make progress. At Waymo Logged some miles in snowy climatesincluding in Truckee, California, Michigan, and upstate New York. It is also being tested in Denver and Seattle, but there is still more work to be done.

Waymo’s fifth-generation system can handle cold weather and light snow, but the sixth-generation Waymo Driver, which is scheduled to be released soon, was specifically designed and tested to handle extreme winter conditions, Chen said.

In addition to better data, Waymo is exploring other solutions to help its automated robot navigate smoother streets. For example, small mechanical wipers have been installed to help remove snow from a lidar sensor located on the roof of the vehicle. More powerful heaters are also included to help defrost all sensors. Waymo’s current system has already been trained for icy road conditions and can handle challenges like black ice, Chen said, citing sub-zero temperatures in Austin last winter. He said every vehicle in its fleet records this data while it is on the road, like a mobile weather station.

“Suppose the car encounters a slippery patch,” Chen said. “In effect, it will send this information to the rest of the fleet, and now the other vehicles in the fleet know that this particular location is slippery.”

If conditions deteriorate to the point where roads become unsafe and most people stay home, Waymo may temporarily discontinue service. Such decisions are rare, but if the public is not traveling, there is no need to keep vehicles on the road. Of course, every city is different: light snow may bring some cities to a halt, while in others, it barely slows traffic.

Even when the snow melts, Waymo can continue to develop its capabilities virtually, using advanced simulation models to simulate rare conditions. This is crucial to address data scarcity issues, Chen said. This is where AI comes into play: generative and foundational models are integrated into the system. Waymo uses layered models that can differentiate between different types of snow — wet, powdery, icy, etc. — and feeds that information back into training pipelines.

It may be a while before Waymo customers experience their first snow robotaxi ride. The company said it will begin operations in Washington, D.C., next year, but has not yet set a date for other East Coast cities. Waymo is also aiming to launch in London and Japan at a later date. But as temperatures drop and the wind begins to blow in the air, Chen and his team will be ready.

“The problem of autonomous driving…is very difficult in itself,” he said. “Now add these crazy weather conditions. It’s a very difficult job.”