Keep the membership. (You don't mention debt or an anemic savings rate, so I assume you and your wife spend a couple of bucks here and there on entertainment and non-essentials.)
You're getting very good usage out of your memberships, so are getting health benefits, time together, and taking awaytime you could spending on non-Mustachian pursuits. At 5-6 visits per week, this is costing you less than $2 per workout. All without having to invest in and maintain a variety of equipment, and saving the cost of your at-home showers (this does add up). Also, group exercise classes (and just the act of going to the gym, a kind of community-gathering place for fitness) can be motivating in a way you won't get by working out just the two of you, at home. (I know you don't use the classes yet, but they're another way to stay motivated, if you ever need them.)
If you're in a debt emergency or just feel the need to cut expenditures, I'd look elsewhere, first. Unless you're ultra-Mustachian already, you're probably still spending money on stuff that hurts you (e.g. eating out, booze, junk food) or has less benefit that exercising (e.g. movies, cable TV, clothes, fancy cell phones, driving your car instead of walking/biking, food/drink/gifts for special occasions, etc.) You might have bigger fish to fry/lower-hanging fruit to grasp at.
Good for you for examining everything, though, and huge congratulations for exercising so much! Keep it up.