Someone at work mentioned Vero Beach as a cheaper place to live. I can't vouch for the safety or surroundings since I haven't spent much time there.
The more expensive areas in South Florida are Dade, Broward, and Palm Beach counties. Once you start getting north of there it gets cheaper.
You can also check out the Melbourne/Cocoa Beach area if you want to stay on the east coast.
Bike riding along A1A along the east coast is nice. I've done some rides starting in Jupiter and going north.