Good job backing off this. I get that feeling, believe me! But the real problem is that you don't know what you'll actually need in the future, or what your priorities will be then. So no reason whatsoever to jump in and spend that kind of money based on untested assumptions.
We decided probably 15 years ago that we'd spend months every year in a specific vacation spot. When the kids were little, we bought a 2Br condo that we could afford, figured we'd rent it out full-time until the mortgage was paid off, then convert to short-term rentals so we could start using it. 10 years later, we had it paid off. But (1) the kids were almost grown and couldn't comfortably share the small second bedroom (teenage boy + girl = significant privacy concerns), and (2) the town had changed the zoning and banned short-term rentals. So just when we were at the point of being able to kick back and enjoy it, that condo no longer worked for what we needed!
So now we're selling it, and we've just bought a different place, with 3 Br and outside of town (and a garage for DH's woodworking hobby). The really good news is that we've saved far more than we had thought possible c. 13 years ago when we originally bought the condo. So even though the kind of place we want now is more expensive than we could have afforded c.2011, we can actually afford that and more without interfering with FI. IOW, we were both stupid (couldn't predict what we'd actually want/need 13 years down the road) and lucky (better-than-expected finances meant we could afford better than we even dreamed of 13 years before); we still get the dream, and it's even better than we thought we could have back when we were fretting about buying someplace, anyplace in that dream locale.
IMO, these sorts of choices should be looked at as consumption vs. "investment." Sure, the house might appreciate. But the reason you want to buy it is to live there and never sell anyway! You can't succeed financially by focusing on appreciation that you don't ever intend to actually realize! And that means the extra money you'd be spending would be for "I get to enjoy a new house in this gorgeous area" (i.e., consumption) not "I'm going to make so much money on this deal I can FIRE a year earlier" (i.e., investment). And for a consumption decision, what really matters is how those extra monthly costs will affect your FIRE plans -- and whether the tradeoff is worth it.