I had no problem carrying a mortgage in retirement, because the cost of borrowing was so low. At 3.125 percent, there is simply no point to paying off the loan. However, in your shoes, I would not use 4 percent as the mortgage rate in any calculations. Rates today are in the high fours, and heading higher. In a more "normalized" environment, 6 percent is what I would expect to see. That's likely to be the case five years out and that's the figure I would use if I were making any decisions about where to live then. Mortgage rates affect house prices, especially outside of the most competitive markets, but the economy is a bigger factor.
Not sure where you are looking in California, but in a competitive market it's extremely difficult to buy with a VA loan. With five offers on the table, a seller is going to choose the offer with the fewest contingencies and the highest likelihood of closing on time. A buyer with a VA loan will be at the bottom of the list. If the market normalizes and is balanced or there are more sellers than buyers, you will have a better chance.
Another problem you will encounter is lending requirements. Lending standards are designed for W-2 wage earners, not early retirees with a stash and an Army pension. Unless you have a history of stable monthly withdrawals from your paper asset accounts, that income will not be counted in qualifying you. Only the pension income will be counted. That will limit the amount of mortgage you will be able to get. There are a few lenders that will do asset based mortgages, but the rates are higher.
It's really early to do detailed calculations of housing costs. You can be precise in your math, but completely inaccurate in your conclusion. In your shoes, I would focus on the barriers I will likely encounter and finding ways to overcome them. For example, can you save more in taxable accounts to up your down payment to 50 percent? Can your wife continue her job at the new location so her W-2 income would be counted in the mortgage qualification calculation?
One more thing. Utility costs are outrageous here. I think the SRP base rate in Phoenix is about 8.7 cents per kilowatt hour. PG&E is two to three times higher. Peak price is around 34 cents per kilowatt hour, last I checked. I would look at your bill and then your in-laws' bill for comparison.