Travel is allocated $6k a year, but we honestly go as necessity dictates (elder care) or as a good opportunity comes up. There's always a limiting factor (vacation time, kid, health) so I don't think we actually spend $6k/year, but I'd have no problem if we did. The kind of enjoyment vacations bring us (anticipation, experience, memories) is worth the cost, to us.
Though flying together would be ideal, would you consider flying separately (so one could connect via the U.S.?) If the savings are there it may be worth doing. Even when flying with a companion so much time on the flight is kind of solitary (sleeping, reading, watching movies) being solo might not feel lonely.
For the one of you needing a visa for the U.S., can you start the process to get it? I know it's a slog, not free, and approval is not guaranteed, but if granted it's good for several years and would open the door to nearby, inexpensive holidays (road trip across the border) and enable you to both fly via the U.S. (possibly less-expensive flights.)