The only thing lower than supply of available homes on the market are mortgage interest rates. Prices are going to stay high for awhile.
Would the primary house at your current property fit your growing family needs? That would free up the guest house to rent out and you wouldn't be spending six figures on a new place plus closing costs, fees, moving expenses, etc.