What you must do is convince him to move up his level of risk-tolerance. Simple, but not easy.
It's not a bad situation if he pays off the house today, given current valuations are elevated, but he'll likely do much better by staying invested and paying it off over time.
Another alternative would be for him to always withdraw 4% and pay any extra cash over his living expenses toward the mortgage. Even maintaining the mortgage for 10 years will be huge.
There is a value to having a home paid off, so my opinion as a 20 something is very different than a new retiree when it comes to debt.
Please let him know, and make sure he understands, that he should think about the situation today toward debt the same as if he were 27 years younger. He still needs his portfolio and finances to last for quite some time, so dialing back risk too far is also a risk in itself.