How far is too far is very cyclist-specific, and of course can be increased. Regardless, 3.5 miles shouldn't be too far for anybody who has any kind of shape. (To give an anecdote, when I first rode 20 miles to work I was as sore as I've ever been doing anything in my life; the second time I was sore, but at least I could still walk; since then things have gotten progressively easier, where now the 40-mile round trip is no big deal.)
But I don't think that answers the heart of your question. Good cycling infrastructure on your route to and from work is critical for an enjoyable ride, in my opinion. I ride bike paths where possible, but otherwise only ride on roads with good shoulders. When lanes get narrow enough to where there is no safe space for a car to path, it turns rather more stressful than enjoyable for me.
Does your city have a cycling website? Mine has a large map of the city showing where designated bike lanes are, which has been key for me finding enjoyable routes.