Humans also work best in relatively predictable environments; That's why freeways see fewer crashes than suburban streets. It's also why we have lane markings, road signs, and rules about what is and is not acceptable - even on an empty road, drivers are expected to stay on the proscribed side, for example.
Well, the key point here is that humans work best in environments that are predictable
relative to human interactions. Robots do not have anything approaching human understanding of how to interact with other vehicles. They are not as good as people at recognizing unsafe behavior in drivers and pedestrians.
I think your understanding of AI is a few decades out of date. It's a very rapidly evolving field, and things that were science fiction twenty years ago are commonplace today.
I wasn't aware that you had expertise in robotics and were able to assess my competence in that subject. What is your experience with AI? Until 2012, the last year of DARPA's Grand Challenge, I was involved in a number of robotics projects with several universities and companies. I'm not aware of any significant advances recently that would invalidate anything I've said here, but I'd be happy to hear your thoughts on why you think I'm decades out of date. I can tell you that command and control voice interfaces have not advanced much beyond directed dialogues, assuming that you know what those are. Absolutely critical to significant progress is the need to improve machine learning strategies, but none of the current approaches scale up to the kind of experience-based reasoning that is needed for human-like reasoning. Real world traffic conditions are too chaotic for current technology, but there are many situations (e.g. in manufacturing) where they lend themselves well to less chaotic environments.
When it comes to a limited and rule-bound activity like driving a car, the human ability to be distracted and to fool itself into imagining that it is aware of far more than it actually is is a liability. A robotic vehicle has significant advantages over a human; It can take in more information from more sources, it never gets tired or emotional, it doesn't fill in gaps in it's awareness with guesses about what ought to be there, it doesn't suffer attentional blindness, and it can communicate its intentions to other road users and negotiate a mutually beneficial set of actions in a way that humans simply cannot.
Have you ever actually interacted with autonomous vehicles? Just for starters, you should know that machines break down over time. So, even if you have a very good system, there are lots of things that can go wrong with it. In working with robots, I've experienced several situations where nobody, even the programmers, understood why the robots were not behaving as expected. Unfortunately, robots lack self-awareness, so it is really difficult to get them to explain why they are doing what they are doing. Programming self-awareness into machines is a very hot topic in the field these days.
One NASA project involving a Mars rover that I once witnessed had a situation where the robot simply froze. It took an hour to figure out that one of the optical sensors had failed, which made it impossible for the robot to carry out a command. It had no way to inform its human controllers that that was what the problem was. In aircraft incidents, many, if not the majority, of problems are caused by the pilot not understanding what the automated systems are doing.
In surveys, over 90% of human drivers rated their own ability as 'above average'. AIs don't kid themselves that they are good enough to break the rules and get away with it. AIs don't try to impress their friends, or the girls, by pulling stupid stunts. AIs don't think that they will probably be fine, because they only had a few beers.
Yup. They aren't human. That is also where their weaknesses lie. They don't reason like human beings and do not handle operation under uncertain conditions as well. Hence, mixing them in with people, who tend to anthropomorphize them, can lead to some serious consequences. Where autonomous vehicles operate around humans, the consequences can be quite disastrous.
Disagree. While we can't truly teach an AI to think we can give it a ranked list of the value of things so it can pick the least harmful collision.
Of course. And those ranked lists depend on the ability of the programmer to imagine what could possibly go wrong. All they have to do is be more knowledgeable about possible collision scenarios than they are about writing sophisticated AI programs. What could possibly go wrong with that? A truly intelligent machine ought to be able to reassess and change its priorities over time. That is, the "ranked list" cannot remain fixed and rigid. We can design programs that do that to a limited extent, but they are really very tricky things to work with.
...While we can't truly teach an AI to think we can give it a ranked list of the value of things so it can pick the least harmful collision.
And crucially, unlike a human driver, it doesn't have to come up with this list 'on the fly', nor does it get paralysed by indecision. An AI choosing whether to run down a human or a dog won't stop to think about how cute the dog is, or how much it looks like its own pet, or that the human is of the wrong race, or wearing a t-shirt supporting a controversial political view...
You see this as an advantage that machines have over humans, but it really isn't. Part of what it means to be intelligent is to have the flexibility to readjust priorities over time--to become better at anticipating and solving problems. Uncertainty is not a bug in human intelligence. It is a feature.
The responsiblity for the decision remains in human hands - but not in the hands of one stressed out human in an unexpected and time constrained situation. And if the comittee that sets the rules and priorities finds that it leads to unexpected bad outcomes, then that can be fixed by a simple software update. Human drivers are much harder to influence to change their bad behaviours.
Have you ever experienced a situation where your computer screen froze or the computer crashed? You've seen a blue screen crash, haven't you? If you have any experience in programming, you know that very complex programs can get "confused", albeit not emotionally upset in the human sense. That reminds me. My Toyota dealer issued a recall for my Prius not long ago. I have to take it in to have the software upgraded. Apparently, my car can suddenly lose power at any time, although such failures are very rare.