The Y runs no joining fee events throughout the year. Ask them if they know when their next one is, but it seems like it is every quarter. I joined in January, and they just finished another during April. There is no contract and you just have to give them 30 days notice for them to cancel your membership.
They also have adjusted fees based off of income. The details should be on their individual site and you can print out the form from there or go in and fill it out. If you are lower income (and folks in FIRE often are), ask about financial assistance. It is a short one page form, and you can show them your tax statement to prove income (they don't take a copy of it at my place, just needed to show them a pdf on my phone verifying my name/address and then showed them the line item of total income MAGI.
I prefer the Y because there is a strong leaning towards older folks and less the grunty/rude body building folks. It's normal people - moms dropping in for some cardio, elderly folks working out of the bikes and chatting around the front area drinking the coffee, some young professionals stopping by on their lunch to hit the weights or swim a few laps. And even the guys and gals that are in the weight section and super ripped aren't as rude as other gyms I've been part of. My husband and I are both noobs at lifting and weight training, I am still working on losing fat so I'm flabby and weak, and we wanted a place where we didn't feel stupid being over in the section trying to figure stuff out. I always felt self-conscious in regular gyms, and don't at the Y for some reason.
At my Y (and the others in the area) the classes are early before work up through noon time, then pick back up for the after work crowd (usually around 6pm to 8 pm). They also run plenty of classes on the weekend. 95% of them are free.
But it sounds like you mainly want to be able to go swim, and the cost factor coupled with the closeness of the other gym may make it more attractive. If they don't make you sign a contract and allow you to pay by card month to month, then I'd likely say they'd work out better for you.