One consideration is private mortgage insurance (PMI). If you borrow more than 80% of the value of the house you will be paying for PMI. So it may make sense to pay your mortgage down to escape the PMI.
I think most scenarios that make sense for paying off the mortgage (instead of investing) emphasize subjective emotional value unique to the person(s) in question over objective market analysis.
In my own situation, my wife and I have decided to put everything we save in the next ~3 years into paying off a substantial mortgage rather then putting into stocks because (1) paying off this concrete cost in our life will reduce our draw when we FIRE (2) my wife has been much more motivated by this goal than by putting money into stocks that bounce around and make her sea-sick. I would say our situation is pretty typical of people that decide to pay off the mortgage rather than invest. What we will not be doing is putting large amounts of money each month towards the mortgage without cash reserves in the event of unexpected job loss that would threaten our ability to continue payments. We will save about 1/3 of what remains on the mortgage (2 year reserve of mortgage payments) then start throwing gobs of cash at the mortgage for 2 years and pay the balance off when we are within striking distance. Of course that leaves open the door to investing elsewhere at any point an opportunity opens up.
So presently we plan to pay off the mortgage and we are saving in a cash account to do it, but our horizon is short, (~3 years), the bond market is crap, and this is an acceptable diversification for our portfolio of ~80% US stock index and ~20% US bond index.
If you have not read it yet, I encourage every potential home owner to look at JL Collins' article about home ownership: http://jlcollinsnh.com/2013/05/29/why-your-house-is-a-terrible-investment/
Best wishes, Aperture.