Potential Pitfalls: Unexpected Events

Extreme Weather

(Washington Post March 16 2015)

Would you support the introduction of a machine that would kill 3,300 Americans a year? The answer is almost certainly no. But what if that technology was a driverless car, and if those 3,300 deaths replaced the roughly 33,000 lives a year that perish on U.S. roads as a result of human error? Is one death caused by a machine error better than 10 deaths caused by human error?

From a utilitarian perspective, it would seem that trading 33,000 deaths for 3,300 would make sense (The 3,300 figure is an arbitrary estimate I’m including for discussion purposes. In theory, self-driving cars will save many lives — exactly how many we don’t know.)

In a keynote address at the SXSW Interactive Festival on Sunday, author Malcolm Gladwell pressed venture capitalist Bill Gurley on our “catastrophically imperfect” network of cars. Gurley honed in on one of the big drawbacks of self-driving cars.

“Humans will be much less tolerant of a machine error causing death than human error causing death,” said Gurley, an early investor in Uber and other disruptive technologies. He describes himself as much more skeptical of driverless cars than most people.

“I would argue that for a machine to be out there that weighs three tons that’s moving around at that speed, it would need to have at least four nines because the errors would be catastrophic,” Gurley said. (Four nines alludes to 99.99 percent, as in the near-perfect safety record self-driving cars may need to gain acceptance. For example, a Web site with “four nines” fails to load only one minute per week.)

Driverless cars may need to be near perfect, but they’ll face a long list of rare circumstances that could be difficult to handle. These unusual circumstances are sometime called edge cases. For example, can a car be programmed to identify an ambulance siren and pull over? Can it respond to an officer directing traffic? What about inclement weather, heavy snow, flooded streets or roads covered with leaves? These things could all disrupt its sensors.

In an panel Saturday at SXSW, University of Michigan professor Ryan Eustice, who is developing algorithms for the maps driverless cars will rely on, acknowledged the challenge.

“To really field this technology in all weather, all kinds of scenarios, I think the public’s been a little oversold to this point,” Eustice said. “There’s still a lot of really hard problems to work on.”

He cited the problem of a driverless car’s sensors being confused by snowflakes during a snowstorm. There’s also the question of whether a driverless car in a snowstorm should drive in its original lane or follow the tracks of the car in front of it?

You might think we can just rely on humans to take over whenever a situation gets dicey. But Eustice and others aren’t fond of that.

“This notion, fall back to a human, in part it’s kind of a fallacy,” Eustice said. “To fall back on a human the car has to be able to have enough predictive capability to know that 30 seconds from now, or whatever, it’s in a situation it can’t handle. The human, they’re not going to pay attention in the car. You’re going to be on your cell phone, you’re going to totally tune out, whatever. To take on that cognitive load, you can’t just kick out and say oh ‘take over.’ ”

He noted how Google had taken the steering wheel and pedal out of its driverless car prototype to avoid the human-machine interface issue, which he considers a huge problem for the field.

In his talk with Gurley, Gladwell noted the surprising disparity between Americans killed in wars and on U.S. roads. It would seem we could do a lot better. But a lot of tough challenges must be solved before U.S. roads can ever become some sort of self-driving utopia.

James Bond's solution ...

Coping With The Unexpected

• Most of us haven’t had a cow wander on to a road in front of us while we’re driving. Still, it’s the type of situation Google is wisely anticipating. A significant challenge for self-driving cars will be handing edge cases, essentially rare situations on the road. As investor Chris Dixon has written, machine learning can quickly solve 80 percent of any problem, but getting the full 100 percent is extremely difficult.
  
  Sure, a self-driving car just drove across the country, but it had the benefit of being on highways, an easy situation for self-driving cars. And the company behind the trip, Delphi, admits the car didn’t do 100 percent of the driving.
  
  It will prove difficult for Google, Delphi or any carmaker to prepare its self-driving cars for all of the odd circumstances the cars will occasionally encounter. Passengers appear very unlikely to be trusted to take over in hairy situations. Google has removed the steering wheel and pedals from its latest prototype, amid concerns that passengers can’t be trusted to effectively take over when necessary.
  
  But a patent Google received last week offers a window into the tech company’s plans for handling these edge cases, such as a few cows in your path.
  
  Google has devised a system for identifying when an autonomous car is in a “stuck position,” and laid out plans for how to get out of the situation. A stuck position is a circumstance where a car can’t get to its final destination without violating some of its rules. For example it might be programmed to only drive on a road and not the shoulder. So what happens when a car breaks down in the lane in front of it?
  
  Google’s patent calls for an assistance center to resolve any situation where the autonomous car can’t follow its planned route. Once the car determines it’s stuck, it will request help from the center, which would rely on a “human expert and/or an expert system” to resolve the issue.
  
  The car sends its location plus data from its sensors. The expert then would suggest a new route, or may request more information, such as images or live video from the car’s cameras to better understand the situation. The patent includes an interface the expert at the assistance center would use. The patent, which leaves many options open for how exactly such a system could work, says the interface also might be controlled by a passenger in the car. Here’s how it could map a route around a group of cows:
  
  
  

  Here the “draw new [trajectory]” button is selected and a new route is drawn to get around the bovine. (U.S. Patent and Trademark Office)
  
  The patent devotes a lot of time to figuring out exactly when a car is stuck and when it isn’t. Google points out that being able to effectively determine when a car is stuck will reduce the demands on expert needed to operate the fleet. The less often experts have to intervene, the lower Google or anyone’s cost to run a network of self-driving cars will be. If the car can’t tell the difference between when it’s really stuck, and when it just needs to be patient, that could be a disaster for Google. Imagine hundreds of self-driving cars stuck in heavy traffic as a concert or sport event lets out, all contacting and overwhelming the assistance center. At the same time, you wouldn’t want a self-driving car stopped behind a double-parked car for five minutes, thinking it’s just stuck in traffic.
  
  Google envisions taking into account a wealth of information before determining if a car is stuck. It will consider the location and the time of day. Is the car near a school as it’s letting out? Or a sports arena shortly after games usually end? Those are situations where a car stuck in traffic might learn it should just be patient.

Start As Taxis In Big Cities

Current limitations in fully self-driving vehicles could make it appealing for Google or other companies to initially roll the technology out as a service, rather than selling cars to consumers.

Thursday at a Volvo self-driving car event in Washington, a California DMV official pressed Google’s Ron Medford, the safety director on its self-driving car project, about when and where the car could drive itself. Because Google is removing the steering wheel and pedals, the human driver in the vehicle would not be able to take control in a difficult situation, such as a snowstorm or heavy rainstorm.

“What you’re developing, if it’s at a situation where it can’t go, what am I supposed to do?” asked California DMV deputy director Brian Soublet during a lively panel discussion. “Just wait for it to clear up so it can go?”

“There are lots of ways we can deploy this technology. You’re talking about vehicle ownership,” Medford answered. “That’s one possibility of a model, but there can be other models in which you wouldn’t own the car, and you’d have other available transportation, if for some reason our car wasn’t able to do it today the way you wanted it to.”

“That will work in a massive urban setting. But does that work in rural Kansas?” Soublet countered.

“Right so, no, it doesn’t work in rural Kansas,” Medford replied. “We’re not ready to service rural Kansas. We’re not ready to go into the snow belt, and we’re talking honestly about what the limitations of the car are.”

The technology behind a fully self-driving car would add a significant expense to a vehicle’s price, which may be a cost customers don’t want to pay, especially when it couldn’t be used in the toughest weather conditions. With a ride-sharing service, an operator could have the vehicles running 24-7. With a fleet of vehicles going nonstop, they’d almost constantly be generating profits and covering the costs of the technology.

Their inability to operate in a heavy snowstorm would be less of a drawback, as consumers wouldn’t be relying on the service exclusively for their transit needs. So far Google has tested its cars in Mountain View, Calif. and Austin, where there’s rarely snow.

Medford stressed it seemed unclear yet what was the best business practice for the technology, and that the current focus is on getting the technology right.

“The car that we are continuing to work on and develop is not one that you can drive everywhere, anywhere, all the time,” Medford said. “I would love for us to take where we are today and leapfrog, and we’ve solved all the problems, but we’re solving many, many of them now. And will continue to solve them.”

His remarks on the business potential echoed what Google co-founder Sergey Brin said in late September at a demonstration for journalists.

“It remains exactly open how we’re going to roll it out,” said Brin, who added that in the near term the upshot of making a service was letting a lot of consumers try it out, and being able to back-up and refine the technology.

What happens if the rules don't work?

• Is it safe to turn left here? A newly surfaced patent application offers a window into how Google’s self-driving vehicles may deal with gray areas.
  
  Left-hand turns are one of the tougher things drivers have to do. They result in far more accidents than right-hand turns, and have been called “the bane of traffic engineers.” The challenge of turning left holds for self-driving cars, too. A Google patent application that was published online last week details a system to assist its self-driving vehicles in difficult situations, such as certain left-hand turns.
  
  Google describes how its self-driving vehicles could reach out to a remote assistant to advise on situations such as right turns on red, temporary stop signs, lane blockages and whether a passenger has gotten in or out of the vehicle. This assistant might be a remote computer, a human in a remote location or even the vehicle’s passenger.
  
  The application describes having a predetermined list of challenging situations so that the remote assistant can receive an early heads-up that its help is about to be needed. That way, the self-driving vehicles should be less likely to get stuck as they await outside intervention.
  
  If the remote helper is a person, she or he would see a live video feed of the situation, plus a representation of what the sensors are gathering.
  
  Some things might be handled by the passenger in the vehicle. This would includes things like whether the car doors are closed, seat belts fastened, and has the car stopped at a good spot for passengers to disembark. While the car attempts to make a right-hand turns on red, a human could confirm that there are no pedestrians about to cross, and that no cross-traffic is approaching.
  
  When a human isn’t helping, Google would rely on its wealth of computing power in the cloud. Huge data centers will be able to process more information than the computers built into a self-driving vehicle. So a decision on the best course of action might be made in the cloud and then relayed to the car.
  
  The application also opens the possibility that Google will use microphones to captures a range of roadway sounds. Right now Google has microphones on its test vehicles that are used to detect sirens from emergency vehicles. The application says a microphone may also be used to capture the sound of the exhaust from vehicles, which could help to identify motorcycles. A Google spokeswoman tells me the microphones are specially tuned for sirens on emergency vehicles and are not used for anything else currently.
  
  Related: what happens if there are animals on the road

Evolving Image-Detection

• The intensity of current research into the field of image recognition reflects the potential ramifications of computers being able to make sense of the visual world, either through neural networks or advances in database classification, or both. There is a magnitude of difference between an AI that can compare a real-world situation to the most prevalent features of a dataset query and one that can itself competently generate such data-sets based on effective learning algorithms – and then use that knowledge.
  
  At the political high end, effective AI-based image recognition has huge significance in terms of security infrastructure, whilst the commercial applications, as currently being researched by Amazon, have significant economic consequences.
  
  Scientific researchers for Facebook AI Research (FAIR) believe that the classic challenges of image classification, edge detection, object detection and semantic segmentation are so near to being solved that the field should turn its sights to the next major challenge: occlusion, or the fact that objects in a photo must often be ‘guessed’, either because they are cropped by the image frame, hidden by other elements, further away from ‘adjacent’ objects than may be immediately obvious or, in certain instances, logically indistinguishable from non-contiguous elements in the frame.
  
  In Semantic Amodal Segmentation [PDF], FAIR researchers Yan Zhu, Yuandong Tian and Piotr Dollár – together with Rutgers University Department of Computer Science fellow Dimitris Mexatas – set small groups of human subjects to the task of ‘completing’ a vector outline for subjects in photographs which are not entirely visible.
  
  In addition to distinguishing the occluded suggested outlines, the volunteers were also tasked with imposing a z-order on the classified objects, i.e. suggesting which are nearer to the camera.
  
  In the case of three huddled fox-cubs, this information is more or less intrinsic due to the fact that the cub with no occlusion (i.e. completely shown) is almost certain to be at the front of the group. In the case of a stag in front of stag-like branches, and regarding a perspective-shortening long lens, or of a musician holding an instrument (see image right), the distinction is far clearer to a human than an AI.
  
  Clutter and clusters in image recognition
  
  At the same time as this paper’s release another research group addresses [PDF] Amazon’s continuing efforts to get robots to accurately choose and pick items from shelves, noting the challenge of ‘clutter’, wherein the detection algorithms applied to the task can easily confuse other objects for their intended object. To this end the Amazon Picking Challenge provides extraordinary visual database resources, along with 3D CAD models that can help the algorithm to reproduce what it is seeing across a variety of potential matches and choose the match that scores highest for comparison.
  
  Amazon is solving less abstract problems than FAIR, however. Though its work may develop principles and techniques that are more widely applicable, its task is primarily concerned with the recognition of ‘Amazon objects’ in an ‘Amazon environment’. The prospect of a thumb over a lens appearing to be a large pink balloon, or of a 750lb gorilla needing to be distinguished from a toy that represents a gorilla, are unlikely to occur and are therefore superfluous to the challenge’s scope.
  
  Facebook’s researchers have wider concerns, almost touching the philosophical at certain points: is a group in itself an ‘object’? A bunch of bananas is a distinct entity in language, for instance, though composed of sub-objects. With more complex subjects such as humans – surely to the fore of Facebook’s scientific interest – the identification of the ‘human object’ leads to immense granularity: gender, age and individual body parts, to begin with, and that’s without addressing contextual challenges such as location, weather and other identifiable objects in the image.
  
  Both the database-driven and the neural-network approaches to image recognition have their limitations, the former of context and the latter of over-extended scope; Amazon seems likely to end up with a ‘baked’ system that works very well but will probably only be of developmental insight to industries that have similar or identical problems. At the same time wider research into object-recognition, particularly in the field of Advanced Driver Assistance Systems (ADAS) for self-driving cars, need to be able to take so many possible variables into account that manual annotation of an imageset-database seems the only realistic route at the present time; even if self-learning Neural Networks could be trusted to learn important information about what they are seeing through their IoT-sensors, adequate computing power for real-time responses in critical situations is not currently feasible.
  
  Regarding the neural approach to image recognition, there is the additional possibility of developing rules which are usually correct but are so likely to fail in particular circumstances as to render them useless in important contexts. If an algorithm begins to understand that similar things can often be found together – such as kittens, bananas and people – it is likely to more successfully understand where there are multiple instances of an object in an image, but may begin to create non-existent ‘groups’ based on the general success of the principle.
  
  Kanizsa-triangleIn their paper the Facebook researchers make note of the Kanizsa Triangle, one of many optical illusions likely to send current image recognition algorithms into a classic Star Trek-style ‘does not compute’ loop. Strictly speaking the image depicted contains six objects, but depicts anywhere between 4-6 objects, depending on your point of view – an interpretive conundrum which is often repeated across image-sets that are generated ad hoc rather than for the purposes of specific database experiments in controlled conditions.