I did it. I was in an outside sales position where my employer was in another state. I had a mandatory office and storage for samples. My adorable 880 sf, 2+1 condo, with a 1-car garage eventually became too small. I sold it and bought a 1700 sf 4+2 townhouse with a generous 2-car garage in a slightly less expensive community. Yes, everything cost more in the new pkace, but a percentage of the increased mortgage, taxes, utilities and insurance were also a write-off. To offset some of the mortgage payment, I rented two of the bedrooms to a roommate for a couple of years, so my OOP wasn't too much higher than for the condo.
In my case, it was mostly the storage space I needed. The size of the office was about the same in each location.
What does your husband do that he "needs" an office? Does it really need to be bigger than you have now? Can that need be demonstrated in case of a tax audit? In your situation, it's hard to see how trading up is the most mustachian choice.