There is no harm in shopping for a new job now. In fact, it may be useful to you in salary negotiations when you're promoted (and if you get competitive offers, may even help you get that promotion). If any offers are as good as the manager position, then why wait?
Your biggest problem is not your lack of the CPA, it's this:
But this whole CPA business is driving me insane. I feel like a failure...
You're thinking that if you walk into an interview, everyone will see what a failure you are. In fact, there are more accountants in industry/government without their CPA than with it.
You know the question will be asked, so you just need to be prepared with an answer that you deliver confidently. Then the interview will move on.
Look up your interviewer before you walk into the room so that you can tailor your response. If it's an accountant with no CPA, then you can say just about anything and you're just confirming their own life choices. If it's some HR flunky, then who cares - they're probably doing behavioral interviewing anyway, and your confidence will speak louder than your words. If it's a CPA, then you just have to not let it shake you.
Maybe you'll get the job, maybe you won't. It doesn't matter. You don't -need- it.