Self driving cars may be less safe than people driven cars

[scottish]

Yes, I know, this is a speculative statement.

My nephew - an engineering student - was over for dinner a few months ago and I was grumbling about self driving cars and the safety issues associated with them.

He commented that "They just have to drive better than the average person."

I had trouble articulating a good argument to this statement.   I pointed out that software implementations tend to have identical failure modes.  So we'll tend to get sets of cars all failing in the same way maybe at the same time.   For example, all self-driving Priuses at the save rev level could fail to work properly in a snow storm, causing (more) mass collisions on the 401.   But this seemed a little speculative even to me.

And then today I was reading an article in the NYT about this very topic.

https://www.nytimes.com/2019/03/23/opinion/sunday/stick-shift-cars.html

I've always been skeptical of automated safety features.   For example, I'd rather have 4 point safety harnesses than airbags.   WTF, who thinks putting little bombs all around the cabin of a vehicle is a good idea?     I think the NYT author does a good job of explaining what bothers me about self driving cars.

In order to get to self driving cars, there are really two approaches.

There's the big bang approach, where we put a self-driving car on the road, perhaps with a (un)safety driver who will take over when the car fails.   As Uber tried to do in Phoenix last year.

Then there's the incremental approach, where we increase the level of automation in the vehicle, allowing massive field tests of new automation by providing assistance to the full time human driver.  Tesla uses this approach, and drivers occasionally allow their vehicles to drive them into stopped trucks and bridge abutments.

So one weak link is the human.    Once you remove the need to concentrate on driving, the driver will tend to, well, not concentrate on driving.

There's tons of technology experience on this forum.   What do you guys think?   Is there a good way to deploy automatic-driving technology?    'cause I don't really see one.

[Abe]

Aside from the self-driving cars, I find the opinion piece irritating mostly because the author is a physician but provides no actual evidence to support his assertions (even though there is some evidence supporting the idea that technology encourages inattentiveness). I'm not sure how this relates to airbags or seatbelt designs since both are well-proven safety features not related to attention span.
But yes, I think that's the problem with any technology that allows for some automation in high-risk situations - humans aren't good at responding to sudden failures in these systems. Unless we have a truly proficient AI that can handle essentially any scenario, we will see accidents due to inattention. However, this is biased by the fact that no one writes articles on how a car prevented a driver who's texting from rear-ending someone on the interstate. That's way less interesting than "Tesla slams into semi-truck, blows up!". This gets back to my first point about the poor quality of this opinion piece due to the author's lack of supporting evidence and actual effort into researching this issue. No one cares about how good he is with a stick-shift.

[gooki]

Tesla and Waymo are doing it right, with different approaches.

Tesla has their AI running on every Tesla on the road all the time, weather autopilot is enabled on the car or not. The AI is continually learning, and reporting when the driver does something different than it would have done. New updates run in this shadow mode until they are confident it’s safe.

Waymo builds simulations based on real world driving experience, then the software is put through millions of miles of simulation for every change before ever being deployed into a vehicle.

The key point here is both methods run through millions if not billions of “real world” scenarios before we even see a minuscule of their capability hits the road. This amount of mileage, and continued improvement is what will make self driving cars safer than humans.

For me the potential safety, way out ranks the potential risk. Self driving vehicles are fully alert 24/7, have an in-depth understanding what is surrounding the vehicle. I cannot say the same for myself. Yes further improvements will need to be made, yes someone will die in a self driving vehicle, but we’re already at the point where the autonomous technology is saving more lives/reducing accidents than are lost/caused.

My real concern is companies taking shortcuts in an attempt to catch up. As we saw with Uber and will likely see again.

[Metalcat]

With the rampant problem of people texting while driving...yeah, I would happily replace everyone with self driving cars for the sake of collective safety.

I simply don't see humans as very safe drivers. Nor does anyone who spends any time in traffic.

[Just Joe]

At least in the USA we seem to be okay with 15,000 deaths a year attributed to firearms. Why are we so concerned about AI driving being perfect? I'm being sarcastic and serious at the same time.

They'll get it right. I see self-driving cars as a moon shot effort. It just takes time. We aren't there yet.

The marketing departments however are building enthusiasm while the engineering departments are still working out the problems.

[GuitarStv]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

[Pizzabrewer]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

[FIRE@50]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I'm gonna disagree with both of these points. There are plenty of examples of "unsafe" things both in your home and outside of it that we all accept. I wouldn't call staircases and swimming pools immoral despite the fact that we know they kill thousands of people every year. We do have regulations regarding handrails and fences however. Taking away the freedom and enjoyment(for some) of owning and operating a car doesn't feel right.

Insurance will certainly change, but again you can still insure your home with all of it's dangers.

[Mississippi Mudstache]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I doubt that. You can get insurance for all kinds of dangerous and unnecessary stuff (FIRE@50 mentioned pools, and I'll add guns to the list). All that's required for someone to offer insurance is a reasonable way to assess the costs, and the auto industry has a century of experience under its belt.

And for what it's worth, I think self-driving cars will have to be much safer than the average human before humans will accept them as an alternative. As in 100x - 1000x safer.

[sherr]

Then there's the incremental approach, where we increase the level of automation in the vehicle, allowing massive field tests of new automation by providing assistance to the full time human driver.  Tesla uses this approach, and drivers occasionally allow their vehicles to drive them into stopped trucks and bridge abutments.

So one weak link is the human.    Once you remove the need to concentrate on driving, the driver will tend to, well, not concentrate on driving.

This was exactly my biggest concern about getting / using Tesla's autopilot upgrade (I know, Teslas are not very Mustacian cars...). I was concerned that it would make me less attentive and less able to respond and that I would be placing too much trust in a system that was not ready to bear it.

However I have found it not to be true in practice. The car's current level of automation frees up my attention from the trivial portions of driving - things like lane keeping, going the right speed, not hitting the guy in front of me - and allows me to pay more attention to the overall situation.

Is there a motorcycle coming up fast behind me? Are there cars currently in my blind spots? Is there an upcoming traffic or merging problem that I'll have be prepared for? I'm more aware of these answers all the time now, not less. And yes, I'm still babysitting the car's lanekeeping and speed adjustments, it just requires a lesser percentage of my attention than doing it myself would.

Yeah people suck and some people will use half-finished self-driving as an excuse to text. But I think those same people would have been bad, inattentive drivers regardless of the car. The question is not just if the car is better than the average driver. I consider myself better-than-average (don't we all) but I think I still benefit from its assistance. But then you add in drunks, texters, or just plain worse-than-average drivers, and it seems obvious to me that self-driving cars will be a huge boon to mankind. I think they are already saving more lives than they cost even at this early stage of development.

Also that stick shift article is particularly unconvincing given that the only evidence he can come up with to support his position is a single study done on ADHD kids. Sure, people who by definition have a medically-diagnosably-hard time paying attention might benefit from a more constant stream of trivialities that force attention. That doesn't make it true for most people in general.

[radram]

I believe costs will drive the change(see what I did there?).

I agree the public is going to have a hard time. They will be taken to the self driving future kicking and screaming.

Specifically, insurance costs will price out human drivers. Insurance companies are great at looking at data and making non-emotional decisions that effect their bottom line.

I believe that as technology improves, costs to insure automated cars will fall, and the human drivers will be seeing huge increases, since they will be the only cars getting in accidents(or as close to near zero as we can get). Finally, you just will not find a carrier that insures human drivers.

[scottish]

Tesla has their AI running on every Tesla on the road all the time, weather autopilot is enabled on the car or not. The AI is continually learning, and reporting when the driver does something different than it would have done. New updates run in this shadow mode until they are confident it’s safe.

Hey is this true?   Do you have a reference?

[GuitarStv]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I'm gonna disagree with both of these points. There are plenty of examples of "unsafe" things both in your home and outside of it that we all accept. I wouldn't call staircases and swimming pools immoral despite the fact that we know they kill thousands of people every year. We do have regulations regarding handrails and fences however. Taking away the freedom and enjoyment(for some) of owning and operating a car doesn't feel right.

Insurance will certainly change, but again you can still insure your home with all of it's dangers.

Allowing someone to drive, knowing that there is both a greater risk of injury and harm to others and no real reason whatsoever beyond vanity to do so is immoral.

[scottish]

Then there's the incremental approach, where we increase the level of automation in the vehicle, allowing massive field tests of new automation by providing assistance to the full time human driver.  Tesla uses this approach, and drivers occasionally allow their vehicles to drive them into stopped trucks and bridge abutments.

So one weak link is the human.    Once you remove the need to concentrate on driving, the driver will tend to, well, not concentrate on driving.

This was exactly my biggest concern about getting / using Tesla's autopilot upgrade (I know, Teslas are not very Mustacian cars...). I was concerned that it would make me less attentive and less able to respond and that I would be placing too much trust in a system that was not ready to bear it.

However I have found it not to be true in practice. The car's current level of automation frees up my attention from the trivial portions of driving - things like lane keeping, going the right speed, not hitting the guy in front of me - and allows me to pay more attention to the overall situation.

Is there a motorcycle coming up fast behind me? Are there cars currently in my blind spots? Is there an upcoming traffic or merging problem that I'll have be prepared for? I'm more aware of these answers all the time now, not less. And yes, I'm still babysitting the car's lanekeeping and speed adjustments, it just requires a lesser percentage of my attention than doing it myself would.

Yeah people suck and some people will use half-finished self-driving as an excuse to text. But I think those same people would have been bad, inattentive drivers regardless of the car. The question is not just if the car is better than the average driver. I consider myself better-than-average (don't we all) but I think I still benefit from its assistance. But then you add in drunks, texters, or just plain worse-than-average drivers, and it seems obvious to me that self-driving cars will be a huge boon to mankind. I think they are already saving more lives than they cost even at this early stage of development.

Also that stick shift article is particularly unconvincing given that the only evidence he can come up with to support his position is a single study done on ADHD kids. Sure, people who by definition have a medically-diagnosably-hard time paying attention might benefit from a more constant stream of trivialities that force attention. That doesn't make it true for most people in general.

The automatic features on my vehicle are limited to the transmission, ABS, cruise control and  EFI/engine management.    EFI and ABS are way better than carburetors and no ABS.     Cruise control is a neutral and I have a mild preference for manual transmissions.

It sounds like you have a Tesla then, is this correct?    This is the first time I've heard the idea that the car can take care of the little picture stuff freeing the driver to worry about the big picture.     

Is anyone else (on the forum) using driving assist features this way?

[scottish]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I'm gonna disagree with both of these points. There are plenty of examples of "unsafe" things both in your home and outside of it that we all accept. I wouldn't call staircases and swimming pools immoral despite the fact that we know they kill thousands of people every year. We do have regulations regarding handrails and fences however. Taking away the freedom and enjoyment(for some) of owning and operating a car doesn't feel right.

Insurance will certainly change, but again you can still insure your home with all of it's dangers.

Allowing someone to drive, knowing that there is both a greater risk of injury and harm to others and no real reason whatsoever beyond vanity to do so is immoral.

What about situations that the self driving car can't handle?    Do you think there'll be a manual mode?    Or are you expecting the automation to be able to do everything except get out and shovel snow?

[GuitarStv]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I'm gonna disagree with both of these points. There are plenty of examples of "unsafe" things both in your home and outside of it that we all accept. I wouldn't call staircases and swimming pools immoral despite the fact that we know they kill thousands of people every year. We do have regulations regarding handrails and fences however. Taking away the freedom and enjoyment(for some) of owning and operating a car doesn't feel right.

Insurance will certainly change, but again you can still insure your home with all of it's dangers.

Allowing someone to drive, knowing that there is both a greater risk of injury and harm to others and no real reason whatsoever beyond vanity to do so is immoral.

What about situations that the self driving car can't handle?    Do you think there'll be a manual mode?    Or are you expecting the automation to be able to do everything except get out and shovel snow?

I'd assume that there will be a manual override for certain situations . . . but I'd also assume that if you cause a death or injury while overriding the automatic mode unnecessarily you will be treated quite harshly in a court of law.

[ixtap]

At least in the USA we seem to be okay with 15,000 deaths a year attributed to firearms. Why are we so concerned about AI driving being perfect? I'm being sarcastic and serious at the same time.

Are we only OK with half of the firearm deaths per year? What percentage of the AI car deaths will we be OK with?

[GuitarStv]

At least in the USA we seem to be okay with 15,000 deaths a year attributed to firearms. Why are we so concerned about AI driving being perfect? I'm being sarcastic and serious at the same time.

What percentage of the AI car deaths will we be OK with?

The percentage of manually driven car deaths - 1.

That's the tipping point where automatic driving becomes empirically better.  But the beauty of automation is that it doesn't regress . . . it gets progressively better and better with each iteration.

[daverobev]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I'm gonna disagree with both of these points. There are plenty of examples of "unsafe" things both in your home and outside of it that we all accept. I wouldn't call staircases and swimming pools immoral despite the fact that we know they kill thousands of people every year. We do have regulations regarding handrails and fences however. Taking away the freedom and enjoyment(for some) of owning and operating a car doesn't feel right.

Insurance will certainly change, but again you can still insure your home with all of it's dangers.

Allowing someone to drive, knowing that there is both a greater risk of injury and harm to others and no real reason whatsoever beyond vanity to do so is immoral.

Oh come on. 100% of drivers die.

More seriously - driving is dangerous. It would be safer if we all took the bus; let's ban driving completely? It just doesn't work like that. Drinking coke makes you obese - we should ban coke.

I'd say that considering how much driving happens we're actually pretty amazingly good at it. Considering everything.

There is risk in going outside your house, there is risk in staying inside it. "But there is a really low chance it could kill me" is not a good reason to not... go for a walk/bike ride/list of other things including driving.

[GuitarStv]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I'm gonna disagree with both of these points. There are plenty of examples of "unsafe" things both in your home and outside of it that we all accept. I wouldn't call staircases and swimming pools immoral despite the fact that we know they kill thousands of people every year. We do have regulations regarding handrails and fences however. Taking away the freedom and enjoyment(for some) of owning and operating a car doesn't feel right.

Insurance will certainly change, but again you can still insure your home with all of it's dangers.

Allowing someone to drive, knowing that there is both a greater risk of injury and harm to others and no real reason whatsoever beyond vanity to do so is immoral.

Oh come on. 100% of drivers die.

More seriously - driving is dangerous. It would be safer if we all took the bus; let's ban driving completely? It just doesn't work like that. Drinking coke makes you obese - we should ban coke.

Taking the bus would be safer, but cars fill a niche for personal transportation that buses do not.

For the bulk of driving that takes place there is zero benefit to manually driving a car over automatically driving a car.

These are completely different scenarios you're comparing.

There is risk in going outside your house, there is risk in staying inside it. "But there is a really low chance it could kill me" is not a good reason to not... go for a walk/bike ride/list of other things including driving.

Agreed, but there is benefit and utility in going out of your home.  There is no benefit in being able to drive manually over being driven in an automated vehicle for the vast majority of uses of vehicles.  Again, completely different scenarios.

[FIRE@50]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I'm gonna disagree with both of these points. There are plenty of examples of "unsafe" things both in your home and outside of it that we all accept. I wouldn't call staircases and swimming pools immoral despite the fact that we know they kill thousands of people every year. We do have regulations regarding handrails and fences however. Taking away the freedom and enjoyment(for some) of owning and operating a car doesn't feel right.

Insurance will certainly change, but again you can still insure your home with all of it's dangers.

Allowing someone to drive, knowing that there is both a greater risk of injury and harm to others and no real reason whatsoever beyond vanity to do so is immoral.

Do you have a staircase in your house? Was it immoral to build that home with a staircase knowing that it is incredibly unsafe and that you could just live in a one story home? I would say no.

[FIRE@50]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Once we get to that point it will be difficult if not impossible to get insurance for self-driving.

I'm gonna disagree with both of these points. There are plenty of examples of "unsafe" things both in your home and outside of it that we all accept. I wouldn't call staircases and swimming pools immoral despite the fact that we know they kill thousands of people every year. We do have regulations regarding handrails and fences however. Taking away the freedom and enjoyment(for some) of owning and operating a car doesn't feel right.

Insurance will certainly change, but again you can still insure your home with all of it's dangers.

Allowing someone to drive, knowing that there is both a greater risk of injury and harm to others and no real reason whatsoever beyond vanity to do so is immoral.

What about situations that the self driving car can't handle?    Do you think there'll be a manual mode?    Or are you expecting the automation to be able to do everything except get out and shovel snow?

I'd assume that there will be a manual override for certain situations . . . but I'd also assume that if you cause a death or injury while overriding the automatic mode unnecessarily you will be treated quite harshly in a court of law.

The purely autonomous cars being tested today don't even have a steering wheel. You would have no chance to override.

[GuitarStv]

Do you have a staircase in your house? Was it immoral to build that home with a staircase knowing that it is incredibly unsafe and that you could just live in a one story home? I would say no.

You appear to be missing the argument.

There is utility in having stairs because they are used to transport you from level to level in a home where elevators are cost prohibitive.  A multi-story home or apartment has utility (it's not possible for everyone to have a one story bungalow given space and money restrictions).
 If there was a cheap, effective stair climbing device that was much safer than using stairs and would work in all instances that stairs are normally found . . . then yes, I'd argue that it would be immoral to continue using stairs.

There is no utility in being able to drive manually if it is safer to drive automated.

The purely autonomous cars being tested today don't even have a steering wheel. You would have no chance to override.

That just means that the bar will be higher for the automated cars before it can be accepted as safer.  They'll have to be proven safer in all weather conditions, all light conditions, all road conditions.

[sherr]

Tesla has their AI running on every Tesla on the road all the time, weather autopilot is enabled on the car or not. The AI is continually learning, and reporting when the driver does something different than it would have done. New updates run in this shadow mode until they are confident it’s safe.

Hey is this true?   Do you have a reference?

Not all cars, just the ones that have purchased the Autopilot upgrade. And it's not "new updates" that are running in shadow mode, but the car's current software version (so while it's useful for AI training / feedback it doesn't help with validating new releases). But yeah the basic idea is accurate, and Tesla talks about it openly.

The automatic features on my vehicle are limited to the transmission, ABS, cruise control and  EFI/engine management.    EFI and ABS are way better than carburetors and no ABS.     Cruise control is a neutral and I have a mild preference for manual transmissions.

It sounds like you have a Tesla then, is this correct?    This is the first time I've heard the idea that the car can take care of the little picture stuff freeing the driver to worry about the big picture.     

Correct, I have a Tesla Model 3.

[dougules]

I believe costs will drive the change(see what I did there?).

I agree the public is going to have a hard time. They will be taken to the self driving future kicking and screaming.

Specifically, insurance costs will price out human drivers. Insurance companies are great at looking at data and making non-emotional decisions that effect their bottom line.

I believe that as technology improves, costs to insure automated cars will fall, and the human drivers will be seeing huge increases, since they will be the only cars getting in accidents(or as close to near zero as we can get). Finally, you just will not find a carrier that insures human drivers.

I don't think all the public will have a hard time with it.  There will be plenty of early adopters that will jump at the chance to use self-driving cars. 

The place where public opinion may be an issue is laws regarding self-driving cars on the roads.  If there is a lot of fear self-driving cars may be stymied by being banned on the roads. There will probably always be jurisdictions that will allow testing to look more tech friendly (ie Pittsburgh). 

It is possible that self-driving cars will be more dangerous initially, but, unlike with human drivers, every incident will prompt improvements. 

[sherr]

Hey is this true?   Do you have a reference?

Not all cars, just the ones that have purchased the Autopilot upgrade. And it's not "new updates" that are running in shadow mode, but the car's current software version (so while it's useful for AI training / feedback it doesn't help with validating new releases).

Actually let me walk some of those restrictions back. It's apparently all "Autopilot Equipped" cars, may or may not be talking about having purchased the Autopilot upgrade. All modern Tesla cars come equipped with the Autopilot hardware, so that may be what they're talking about. (FYI you can turn data collection off in the settings if that's a concern).

And according to the Version 9.0 software announcement they *do* actually use shadow mode to verify the safety of new features before making them available.

[driftwood]

At least in the USA we seem to be okay with 15,000 deaths a year attributed to firearms. Why are we so concerned about AI driving being perfect? I'm being sarcastic and serious at the same time.

Are we only OK with half of the firearm deaths per year? What percentage of the AI car deaths will we be OK with?

•In 2016, 10,497 people died in alcohol-impaired driving crashes, accounting for 28% of all traffic-related deaths in the United States.1 (CDC website)

For 2016 specifically, National Highway Traffic Safety Administration (NHTSA) data shows 37,461 people were killed in 34,436 motor vehicle crashes, an average of 102 per day.[1] (Wikipedia)

I think general American culture is that we're completely fine with thousands of deaths as long as our personal liberties aren't limited. Just look at people who will aggressively drive during high traffic times just to get a car length or two ahead... they aren't getting to their destination much faster, maybe a matter of seconds or a minute, but they're still willing TO RISK THEIR LIVES for that feeling of control. And if you drive the speed limit, you're the asshole, and people will claim that you're even a danger to other drivers because you don't want to drive faster and break the law.

The soda example is true too... you can't make eating junk food illegal, even if poor eating decisions kill tens of thousands of Americans a year. Don't you dare step on my freedom!

Insurance being a driver of accepting self-driving cars does make sense though, because the collective power of insurance companies can overcome the control culture of the average driver.

[Rufus.T.Firefly]

Do you have a staircase in your house? Was it immoral to build that home with a staircase knowing that it is incredibly unsafe and that you could just live in a one story home? I would say no.

You appear to be missing the argument.

There is utility in having stairs because they are used to transport you from level to level in a home where elevators are cost prohibitive.  A multi-story home or apartment has utility (it's not possible for everyone to have a one story bungalow given space and money restrictions).
 If there was a cheap, effective stair climbing device that was much safer than using stairs and would work in all instances that stairs are normally found . . . then yes, I'd argue that it would be immoral to continue using stairs.

There is no utility in being able to drive manually if it is safer to drive automated.

The purely autonomous cars being tested today don't even have a steering wheel. You would have no chance to override.

That just means that the bar will be higher for the automated cars before it can be accepted as safer.  They'll have to be proven safer in all weather conditions, all light conditions, all road conditions.

There are countless of activities that are not immoral yet provide zero utility and would result in some unfortunate accidents every year. We allow people to skydive yet there is zero utility in jumping out of a perfectly good airplane.

This argument is akin to all motorcycles are immoral because cars are safer and perform the exact same utility.

I'm all for self-driving car technology and welcome it's advancement, especially as it becomes significantly safer than manually-driven cars. However, there are some who enjoy driving as recreation and they'll be around long after self-driving cars are vastly safer than the alternative. Driving a car will then fall into a perfectly legal and valid hobby activity - much like equestrian, flying a private plane, hunting, or any other thousands of activities which have a non-zero chance of harming the person doing them and innocent bystanders.

[GuitarStv]

Do you have a staircase in your house? Was it immoral to build that home with a staircase knowing that it is incredibly unsafe and that you could just live in a one story home? I would say no.

You appear to be missing the argument.

There is utility in having stairs because they are used to transport you from level to level in a home where elevators are cost prohibitive.  A multi-story home or apartment has utility (it's not possible for everyone to have a one story bungalow given space and money restrictions).
 If there was a cheap, effective stair climbing device that was much safer than using stairs and would work in all instances that stairs are normally found . . . then yes, I'd argue that it would be immoral to continue using stairs.

There is no utility in being able to drive manually if it is safer to drive automated.

The purely autonomous cars being tested today don't even have a steering wheel. You would have no chance to override.

That just means that the bar will be higher for the automated cars before it can be accepted as safer.  They'll have to be proven safer in all weather conditions, all light conditions, all road conditions.

There are countless of activities that are not immoral yet provide zero utility and would result in some unfortunate accidents every year. We allow people to skydive yet there is zero utility in jumping out of a perfectly good airplane.

Agreed.  In this case though, the difference is that the danger is to the person performing the activity.  With cars, far too often the danger is to others.

This argument is akin to all motorcycles are immoral because cars are safer and perform the exact same utility.

Actually, I'd argue that cars do not perform the same utility as motorcycles.  They are heavier, and consume more fuel.  I'd assume that self driving motorcycle rules would soon follow self driving car rules though.

I'm all for self-driving car technology and welcome it's advancement, especially as it becomes significantly safer than manually-driven cars. However, there are some who enjoy driving as recreation and they'll be around long after self-driving cars are vastly safer than the alternative. Driving a car will then fall into a perfectly legal and valid hobby activity - much like equestrian, flying a private plane, hunting, or any other thousands of activities which have a non-zero chance of harming the person doing them and innocent bystanders.

I've got no problem with people who want to drive recreationally.  They can do so on safe closed courses.  To allow them to put other's lives at risk by playing around on the road is immoral though.

After all, equestrian activities don't take place on highways.  You must get special licensing and follow an awful lot of rules to fly a private plane in public airspace.  Ditto for hunting, etc.

[FIRE@50]

Very few people are driving the safest car commercially available today. Does that mean they are all immoral for doing so?

[GuitarStv]

Very few people are driving the safest car commercially available today. Does that mean they are all immoral for doing so?

How do you define 'safest car commercially available today'?

Safest for occupants?  Safest for other party in the case of an accident?  Safest in terms of electronic features like backup camera?  Mechanically safe?  My response would change depending on your definition.

Assuming the latter though, I'd argue that if you are driving a car on the highway with brakes or steering that don't work properly, yeah . . . you're being immoral.

[driftwood]

Very few people are driving the safest car commercially available today. Does that mean they are all immoral for doing so?

How do you define 'safest car commercially available today'?

Safest for occupants?  Safest for other party in the case of an accident?  Safest in terms of electronic features like backup camera?  Mechanically safe?  My response would change depending on your definition.

Assuming the latter though, I'd argue that if you are driving a car on the highway with brakes or steering that don't work properly, yeah . . . you're being immoral.

but this isn't an issue of morality really... there's are tons of immoral things we do that have an impact on ourselves and others and oftentimes the immorality of it is culturally acceptable.

Speeding is immoral but if you go drive at or under the speed limit you'll see hundreds of cars passing you... each one of those drivers is totally ok with breaking the law because they have a justification for it. Morality isn't really as important as most pretend it is.

[sherr]

Speeding is immoral but if you go drive at or under the speed limit you'll see hundreds of cars passing you... each one of those drivers is totally ok with breaking the law because they have a justification for it. Morality isn't really as important as most pretend it is.

Speeding is illegal. Immoral does not equal illegal.

If we're approaching the question from a least-danger-to-others framework then not speeding / not driving at the rate of traffic could be considered immoral.

[driftwood]

Speeding is immoral but if you go drive at or under the speed limit you'll see hundreds of cars passing you... each one of those drivers is totally ok with breaking the law because they have a justification for it. Morality isn't really as important as most pretend it is.

Speeding is illegal. Immoral does not equal illegal.

If we're approaching the question from a least-danger-to-others framework then not speeding / not driving at the rate of traffic could be considered immoral.

Then we get into the gray realm of what is 'right'. I personally reject the idea that a group of people can, while breaking the law, pressure me into also breaking the law in the interest of safety. What is safe is for everyone going faster than the speed limit to slow down. I'd argue that is also the moral and right thing to do. But I know that you can't expect Americans to do that, so we get into this argument about how I should break the law to be a 'safe' driver.

Anyways, if self driving cars stick to the legal speed (which might very well be much higher in the future if they can handle increased speeds and decreased reaction times), this will be a moot point.

[sherr]

Speeding is immoral but if you go drive at or under the speed limit you'll see hundreds of cars passing you... each one of those drivers is totally ok with breaking the law because they have a justification for it. Morality isn't really as important as most pretend it is.

Speeding is illegal. Immoral does not equal illegal.

If we're approaching the question from a least-danger-to-others framework then not speeding / not driving at the rate of traffic could be considered immoral.

Then we get into the gray realm of what is 'right'. I personally reject the idea that a group of people can, while breaking the law, pressure me into also breaking the law in the interest of safety. What is safe is for everyone going faster than the speed limit to slow down. I'd argue that is also the moral and right thing to do. But I know that you can't expect Americans to do that, so we get into this argument about how I should break the law to be a 'safe' driver.

That's not really my point. I'm not advocating for speeding. I'm just pointing out that you can't make the unsubstantiated claim that "speeding is immoral".

To respond to your actual point, I would argue that regardless of what you think is immoral most people obviously do not think that speeding is immoral, hence why they do it. If there is a dramatic perception shift around how much you endanger others by not using a self-driving car then that might not be viewed in the same category as speeding for forever.

I'm more interested in talking about OP's topic though. And to that I say that I think the future of self-driving cars is both bright and inevitable.

[robartsd]

I'm sure liability insurance for human drivers will continue for quite a while. If self driving cars improve overall road safety, than insurance rates should even go down (though decreased experience driving might make humans poorer drivers than they are now increasing rates). Eventually, it will likely be cheaper to upgrade to self-driving vehicles than to continue to insure human drivers which will probably cause many people to give up driving, but the option is likely to continue to exist as a luxury. Then we'll start to see restricted access roadways that only permit autonomous vehicles and gradually human drivers will become a novelty.

[EricEng]

However I have found it not to be true in practice. The car's current level of automation frees up my attention from the trivial portions of driving - things like lane keeping, going the right speed, not hitting the guy in front of me - and allows me to pay more attention to the overall situation.

Is there a motorcycle coming up fast behind me? Are there cars currently in my blind spots? Is there an upcoming traffic or merging problem that I'll have be prepared for? I'm more aware of these answers all the time now, not less. And yes, I'm still babysitting the car's lanekeeping and speed adjustments, it just requires a lesser percentage of my attention than doing it myself would.

I feel exactly the same.  I find driving an older car stressful and draining.  After 2 hours I'm wiped, my nerves are shot from monitoring and maintaining every little thing.  However, we recently got a Honda with lane keeping and adaptive cruise control.  I did a 5 hour non stop drive with it and came out feeling fine, could have done hours more.  I still had to monitor the road, but I didn't have to focus hard with 100% attention.  It was fine with 30-70% depending on conditions and let me focus on the bigger picture as you said.  I'm ecstatic for the future of automated cars.

[Indexer]

There's the big bang approach, where we put a self-driving car on the road, perhaps with a (un)safety driver who will take over when the car fails.   As Uber tried to do in Phoenix last year.

A very important part of that story that is often overlooked was the fact that the car's factory equipped automatic braking system detected the bicycle, but it couldn't brake because Uber disabled it. They thought the auto braking would interfere with the testing of their self driving technology. Uber made the car less safe.

In addition, Uber hired unqualified people to drive these things and the employees watched movies on their phones instead of monitoring the car. Uber created a perfect storm. It's really surprising they didn't have a crash sooner.


I think Waymo would be a much better example of a company that is taking self driving technology seriously.

[scottish]

At least in the USA we seem to be okay with 15,000 deaths a year attributed to firearms. Why are we so concerned about AI driving being perfect? I'm being sarcastic and serious at the same time.

What percentage of the AI car deaths will we be OK with?

The percentage of manually driven car deaths - 1.

That's the tipping point where automatic driving becomes empirically better.  But the beauty of automation is that it doesn't regress . . . it gets progressively better and better with each iteration.

I don't think this point about technology regression is very accurate.   There are a couple of problems.

First - inadequate work on the next iteration.   The Boeing 737 max 8 appears to be an example of this.   A new flight safety features tilts the plane into a dive if one of the sensors fails.    The pilots aren't able to easily disable the feature.

Second - the self driving technology is proprietary to each manufacturer.    There's no knowledge sharing happening so that Tesla can learn from Waymo's experience (for example).

Third - there seems to be no independent oversight of the technology development.   So we're trusting the manufacturers to do a good job without verifying that they are in fact doing a good job.

[GuitarStv]

At least in the USA we seem to be okay with 15,000 deaths a year attributed to firearms. Why are we so concerned about AI driving being perfect? I'm being sarcastic and serious at the same time.

What percentage of the AI car deaths will we be OK with?

The percentage of manually driven car deaths - 1.

That's the tipping point where automatic driving becomes empirically better.  But the beauty of automation is that it doesn't regress . . . it gets progressively better and better with each iteration.

I don't think this point about technology regression is very accurate.   There are a couple of problems.

First - inadequate work on the next iteration.   The Boeing 737 max 8 appears to be an example of this.   A new flight safety features tilts the plane into a dive if one of the sensors fails.    The pilots aren't able to easily disable the feature.

Second - the self driving technology is proprietary to each manufacturer.    There's no knowledge sharing happening so that Tesla can learn from Waymo's experience (for example).

Third - there seems to be no independent oversight of the technology development.   So we're trusting the manufacturers to do a good job without verifying that they are in fact doing a good job.

Yeah, I totally agree on the third point.  This needs to be regulated by governments for it to ever be safe.  The example of Boeing is a great one that shows how well corporate self-regulation and testing works.  Or doesn't.  (I've spent some time working with Boeing engineers . . . it wouldn't be my choice to let 'em loose with no oversight).

Proprietary technology is not really a big deal, but I envision something like an independent set of tests that are government run which need to be successfully completed before a car is certified safe.  Similar idea to what goes on with crash tests to achieve a certain minimum level of safety.

[ender]

imo long haul trucking will be a prime candidate for this first.

You have far less city driving, far more long stretches. There are multiple economic factors driving it.

[LennStar]

Self-driving cars just have to be safer than the average person on average.  As soon as that happens, it would be immoral to allow people to drive for themselves any more . . . because you know that it will cause more accidents/deaths/injuries than fully automated vehicle fleets.

Then there's the incremental approach, where we increase the level of automation in the vehicle, allowing massive field tests of new automation by providing assistance to the full time human driver.  Tesla uses this approach, and drivers occasionally allow their vehicles to drive them into stopped trucks and bridge abutments.

I think there is the answer already: Per mile driven Teslas and other autonomous cars are safer than other cars already.
And they will get better, while the human will not.

What about situations that the self driving car can't handle?    Do you think there'll be a manual mode?    Or are you expecting the automation to be able to do everything except get out and shovel snow?

For the foreseeable future there will always be a manual mode. I cannot see Teslas driving on a car train for example, or onto a car transport ship with <20cm between each car.

Do you have a staircase in your house? Was it immoral to build that home with a staircase knowing that it is incredibly unsafe and that you could just live in a one story home? I would say no.

I would argue that regardless of what you think is immoral most people obviously do not think that speeding is immoral, hence why they do it.

Some people seem to be troubled by the fact that something can be immoral and still be done happily ;)

A new flight safety features tilts the plane into a dive if one of the sensors fails.

There is a feature that would alarm if the two sensors make different data. That is a feature you have to pay extra though.
THAT is the real danger. And whoever was the person responsible for that financial decision should get into jail for manslaughter. It is bad enough for e.g. Tesla to put in the same battery but disable parts of it of you don't pay. But safety features that exist?!?

[dougules]

At least in the USA we seem to be okay with 15,000 deaths a year attributed to firearms. Why are we so concerned about AI driving being perfect? I'm being sarcastic and serious at the same time.

What percentage of the AI car deaths will we be OK with?

The percentage of manually driven car deaths - 1.

That's the tipping point where automatic driving becomes empirically better.  But the beauty of automation is that it doesn't regress . . . it gets progressively better and better with each iteration.

I don't think this point about technology regression is very accurate.   There are a couple of problems.

First - inadequate work on the next iteration.   The Boeing 737 max 8 appears to be an example of this.   A new flight safety features tilts the plane into a dive if one of the sensors fails.    The pilots aren't able to easily disable the feature.

...

For this it just has to be better than a human.  How many accidents have been pilot error? 

Also, the Boeing 737 Max issues are prompting a whole lot of scrutiny.  There will be changes before those aircraft leave the ground again.  If the accidents had been attributed purely to pilot error I don't think we would have as much expectation of correcting the issues.

[scottish]

At least in the USA we seem to be okay with 15,000 deaths a year attributed to firearms. Why are we so concerned about AI driving being perfect? I'm being sarcastic and serious at the same time.

What percentage of the AI car deaths will we be OK with?

The percentage of manually driven car deaths - 1.

That's the tipping point where automatic driving becomes empirically better.  But the beauty of automation is that it doesn't regress . . . it gets progressively better and better with each iteration.

I don't think this point about technology regression is very accurate.   There are a couple of problems.

First - inadequate work on the next iteration.   The Boeing 737 max 8 appears to be an example of this.   A new flight safety features tilts the plane into a dive if one of the sensors fails.    The pilots aren't able to easily disable the feature.

Second - the self driving technology is proprietary to each manufacturer.    There's no knowledge sharing happening so that Tesla can learn from Waymo's experience (for example).

Third - there seems to be no independent oversight of the technology development.   So we're trusting the manufacturers to do a good job without verifying that they are in fact doing a good job.

Yeah, I totally agree on the third point.  This needs to be regulated by governments for it to ever be safe.  The example of Boeing is a great one that shows how well corporate self-regulation and testing works.  Or doesn't.  (I've spent some time working with Boeing engineers . . . it wouldn't be my choice to let 'em loose with no oversight).

Proprietary technology is not really a big deal, but I envision something like an independent set of tests that are government run which need to be successfully completed before a car is certified safe.  Similar idea to what goes on with crash tests to achieve a certain minimum level of safety.

I think the proprietary aspect is a big deal in the long run.    As new techniques and methods are developed, they need public scrutiny.   There's alot of synergy (I hate that word, but there you are) to be had by opening the kimono and allowing public evaluation of how each technique works.

If nothing else, society deserves the safest self-driving technology available & car companies should not be allowed to keep them confidential.

[Holyoak]

Jesus, as much as I like the idea of self-driving vehicles, I'm really having trouble seeing it go full tilt in snowy/icy area's, or the worst, snowy with hills...  Like the testing they did in Pittsburgh.  I don't like driving all that much in snow (I'm in lake effect the snow-belt), and I sure as hell can't see me at this time, being in a SD vehicle in those type of conditions.  Any links of how they deal with the issues I raise?

[Linea_Norway]

Jesus, as much as I like the idea of self-driving vehicles, I'm really having trouble seeing it go full tilt in snowy/icy area's, or the worst, snowy with hills...  Like the testing they did in Pittsburgh.  I don't like driving all that much in snow (I'm in lake effect the snow-belt), and I sure as hell can't see me at this time, being in a SD vehicle in those type of conditions.  Any links of how they deal with the issues I raise?

+1

I think self driving cars need roads with good lines on the sides and in the middle. And probably not too many potholes. This excludes most of Norway's smaller roads and practically all the roads in the winter.

You might want to follow the project they are going to do on Svalbard (Spitsbergen). I think they just implemented a self-driving buss there. And you could say they have quite snowy conditions there. I wonder whether the buss will be able to distinguish a polar bear from a pile of snow? The non so representative thing it that there is only 1 road there, so maybe they will just use beacons, like they have for robovacs.

[Shane]

Although it's not yet a "self-driving car," a couple of days ago my wife and I test drove a 2019 Subaru Outback, which had some pretty cool AI features, like adaptive cruise control, lane keeper assist, blind spot detection, pre-collision automatic braking, etc. It was pretty amazing! We're planning on purchasing the car in late May and keeping it for ~10 years. Pretty sure it'll be the last car we ever buy that has to be driven by a human.

[gooki]

Jesus, as much as I like the idea of self-driving vehicles, I'm really having trouble seeing it go full tilt in snowy/icy area's, or the worst, snowy with hills...  Like the testing they did in Pittsburgh.  I don't like driving all that much in snow (I'm in lake effect the snow-belt), and I sure as hell can't see me at this time, being in a SD vehicle in those type of conditions.  Any links of how they deal with the issues I raise?

The great thing about self driving cars is if the conditions are unsafe, none of them will attempt to travel. Yes it’ll inconvenience some people, but it’ll be a whole lot safer than what we have today with morons driving in conditions they are not equipped to do so safely.

[gooki]

Hey is this true?   Do you have a reference?

Not all cars, just the ones that have purchased the Autopilot upgrade. And it's not "new updates" that are running in shadow mode, but the car's current software version (so while it's useful for AI training / feedback it doesn't help with validating new releases).

Actually let me walk some of those restrictions back. It's apparently all "Autopilot Equipped" cars, may or may not be talking about having purchased the Autopilot upgrade. All modern Tesla cars come equipped with the Autopilot hardware, so that may be what they're talking about. (FYI you can turn data collection off in the settings if that's a concern).

And according to the Version 9.0 software announcement they *do* actually use shadow mode to verify the safety of new features before making them available.

Thanks for taking the time to clarify my comment. FWIW, all autopilot equiped cars is all Tesla’s sold in the last 5 years or so, irrespective of the owner having purchased the Autopilot option.

[Linea_Norway]

Although it's not yet a "self-driving car," a couple of days ago my wife and I test drove a 2019 Subaru Outback, which had some pretty cool AI features, like adaptive cruise control, lane keeper assist, blind spot detection, pre-collision automatic braking, etc. It was pretty amazing! We're planning on purchasing the car in late May and keeping it for ~10 years. Pretty sure it'll be the last car we ever buy that has to be driven by a human.

We have a 10 year (165.000 km) old Subaru Outback that doesn't have all these modern features. But it is currently driving very well.