Never say never and all that but I really have trouble seeing how there is continual orders of magnitude room for improvement in driving AI. Maybe my vision of the end state is hopelessly constrained (hope so!) I dont see how driving AI can double in 'goodness' every year for decades on end
Henry Ford supposedly said "If I had asked my customers what they wanted, they would have said a faster horse." Why is your vision of future AI driving constrained to approaching or incrementally improving the safety rates of human drivers? There are so many other things wrong with cars that AI can potentially fix.
Imagine a world in which nobody owns a personal vehicle, but your smartphone can summon one to your current location in 30 seconds. That vehicle travels at 150 mph without your supervision, and can deliver you to exactly where you need to go, not to a parking lot. Oh, and it generates no carbon pollution, is totally silent, comes to your door pre airconditioned or heated and configured to your preferences for movies or music or workspace, and it can be a convertible or a moving truck depending on what you need today, and oh yes it's free to use.
Imagine a world without parking lots and roads taking up 85% of the surface area of every downtown district. There are no more traffic lights, because those are archaic devices designed to let clumsy and slow humans negotiate intersections and computers can instead negotiate intersections at full speed while communicating with each other at the speed of light. There are no more traffic jams or rush hours.
Imagine a world in which every item is delivered to your door. Groceries, furniture, the daily mail, everything you buy from places like amazon. You don't ever need to go to the store again, unless you want to, because it's literally cheaper and faster to have everything delivered by robots. Stores are just showrooms, places to go look and feel products, places to "shop" but not necessarily to buy.
Urban sprawl stops as less land is devoted to vehicles. Maintenance and mechanical issues are automatically diagnosed and repaired. There is no more road rage. Air quality improves. Drunk driving is non existent. Innocent bystanders never die in a high speed police chase. Houses are redesigned with useful space or storage, instead of 3 car garages. No one is exposed to gasoline fumes and nobody dumps motor oil down storm drains.
The world is a better place with AI-controlled cars. Having them learn to drive is just the first baby step in that process.