I don't know much about a 403b, but do you have a matching amount that you have to contribute to receive additional funds from your employer? If so, do the match and put the remainder into a standard, taxable brokerage account. Start your Roth ladder, if you can, and contribute the annual limit for both yourself and your wife, if you feel like it; but any funds you would put towards your mortgage, put into the taxable account. If push comes to shove, you can liquidate the brokerage account to pay your mortgage off in a lump sum, without encountering the tax bracket hit that you would get by paying your mortgage with a distribution from your 403b.
And yes, you can withdraw qualified distributions from your 403b after age 55 if and only if you separated from that employer most recently and after your 55th birthday. You can't do this trick to any form of IRA, nor to any employer sponsored retirement account except the most recent employer's plan. For example, if you have more than one 403b, or a 401k from a previous employer, you cannot withdraw funds from either of those accounts till 59.5 years old, without taking a 10% penalty tax.