The problem with saying this is that drivers will be fine no matter what happens. Pedestrians and cyclists die.
This is a banal truism. It doesn't get you anywhere logically, unless your proposal is to strip every speed limit down to, say, 25km/h (the level at which it is impossible to die from a collision because the forces are too minor). Since no one, not even you, is proposing that, the situation comes down to a mix of balancing various interests and personal responsibility.
You should encounter very few pedestrians and cyclists on freeways, so there wouldn't be any reason to lower speed limits in these areas as the benefit would be negligible. Where there is little pedestrian traffic and sidewalks/bike lanes are set far back from the road I think a reasonable case can be made for higher speeds.
In urban and suburban areas with extremely high foot and cycling traffic though, reducing speeds increases safety for minimal negative impact. This is where I'd suggest that it makes sense to perform the speed reduction. Besides the safety factor, it would encourage more cycling and less dependency on vehicles for transport. This is the general approach that the Netherlands has used in many places to great success.
You are right that drivers bear most of the moral culpability when it comes to accidents (this does not excuse pedestrians from being careless, though). My solution would be to increase penalties for drivers who do cause accidents. Right now the penalty for speeding is not much less severe than the penalty for speeding and hitting someone (as long as you don't kill that person, and as long as there are no aggravating factors like being drunk). In a more just world, the penalties would scale for actually causing damage. And instead of a fine (which irresponsible people won't pay), the penalties would be mandatory community service and re-education for even 'minor' accidents.
This allows some leeway for responsible drivers to drive to a responsible speed limit (say, 50km/h) rather than requiring all drivers abide by the lowest common denominator speed limit.
To this, I'd argue that a responsible driver would already be operating his or her vehicle at below the limit in the scenarios I suggest limiting speed, so there should be no impact on responsible drivers. You appear to be arguing that just because we're used to the danger it somehow means that driving at 50 kph on dense streets with plenty of vehicular, pedestrian, and cycle traffic makes sense - I'd like to see the data being used to support these claims.
Also, the WHO doesn't agree with your numbers:
"A car travelling at 50 km/h will typically require 13 metres in which to stop, while a car travelling at 40 km/h will stop in less than 8.5 metres." -
This includes braking distance and reaction time. It is not just braking distance (the distance required from the first application of brakes).
A number that fails to account for human reaction time when judging safety of a human operating a vehicle would appear to be a fundamentally dishonest amount to give.
We can't cotton wool the world.
Agreed - this is why I think we should stop giving preference to fragile and complain-y automobile drivers at the cost of the lives of pedestrians and cyclists. Drivers need to toughen up and learn to deal with the extremely minor inconvenience that their choice of transportation sometimes brings.
