I replaced the speakers in my car a couple years ago. For around $60 (open box model) they sound as good as any upgraded stereo that people might pay thousands for in a new car. I only did the front, and the rear is disabled with the fade control, but it's not noticeable in the driver's seat. Bass is lacking, but who cares?
I have no need for a head unit. An aux jack, volume control, and amplifier is all I need.
I was also disappointed in the sound quality from an FM adapter. There's not enough bandwidth to get good sound, and FM broadcast stations have better equipment to make do with what they have than a tiny car transmitter. Bluetooth A2DP is a bit better, but it uses lossy compression and it's noticeable sometimes.
I'm not sure why driving with headphones in is illegal. If you had a soundproof luxury car with the stereo turned up you would hear just as little of your surroundings as having headphones on. In fact, I realized that I can get the same quietness of a luxury car by putting in earplugs and turning up the stereo. The SNR could be the same (depending on the earplugs and the luxury car) in either case, so audio quality would improve and outside noises would be attenuated by the same amount. I tried it once and it was kind of cool, but it also felt weird.