You'll have to color me skeptical about this whole self-driving car thing. I would certainly rather own a self-driving car, and I typically commute by motorcycle so feel it would probably be safer for me if cars were self-driving. Then people could shove their smartphones down their throats without it being a risk to me. But I still think there are too many details that need to be worked out that make the whole thing further off than is advertised. Just today leaving from the gym on the Ducati I was playing through all the steps that would be required if I was commuting to work via SD car. There just seem to be too many things that have to go right - from navigating a parking lot to collision avoidance, dealing with road markings, faded or missing lines, GPS irregularities, other people, training/licensing, etc. Maybe it's just the engineer in me, but I have trouble envisioning this sort of thing with our current road system.
As an engineer as well (software), I highly suggest looking at some of the interface that Self Driving cars use. A fair amount of it is shown in this Ted Talk (
link) and if the engineer part of your brain is anything like mine, you will pause on that interface and look at all the things it takes into account. It's insanely awesome.
As far as the challenges go, there are a lot of them. If we had consistent infrastructure, or were willing to rebuild it, the tech from about 10 years ago would have made it possible. However, our infrastructure is not consistent. One of the things I really like about the SD cars is how they kind of gave up on designing decent infrastructure, and coding for what is (worst case) there, and not what should be (best case). I'd highly suggest watching the whole Ted Talk, but I took the liberty of finding the interface part. Here is the link:
https://youtu.be/tiwVMrTLUWg?t=7m50s.
I totally understand the concerns, and I thought the same way until I started really looking into it. I was thinking how big of a pain in the ass it would be to account for all of those variables. And then I looked at some of the things they are doing, and OMG they are accounting for them all!! I still don't know quite how they would deal with the faded lines, but I do know they are able to identify the signs and road markings (even last year, they were able to identify cones and what they signify). They actually just made their vision API available through GCP, and even the free version can read all the common signage in english (I suppose that starting with facial recognition makes something like a sign pretty freaking easy).