Traffic on 24 is bad, and getting worse. Whether that matters for you depends on how often, and when, you'll need to get into Colorado Springs. I'd suggest staking out the road on the days of the week and times of day you'd need to be driving, and see how it is.
I really like Manitou, seems like it would be a nice place to live.