I know someone that does this, and I think it is very feasible. Once you get up to speed on CPE and general tax knowledge, I would reach out to local or regional firms during the off season and just do some informational interviews to learn how they handle the influx of work. I personally would prefer to work for a firm than to run my own shop- it's just easier to hop in and out, no worries about finding clients, etc. I'm sure your own little practice would be more lucrative, but it would also be a lot more worry.
I'm a CPA but I work in industry (a non-profit). I did a couple years in public before deciding I actually wanted to have a life outside of work. I could see going back to it for tax season, though.