The more you research this question the more you'll find it's as much about emotions as finances. Basically, you are investing on margin. If the investments do well then you will come out ahead and be happy. If they lose, then you'll wish you'd paid down the mortgage.
Facing a declining market that I mistook for the top of the recovery and what seemed like imminent layoffs in my department, I paid off my mortgage so that I wouldn't be forced to do it later when the market was even lower. I'd run the numbers and it seemed like a wash financially given the poor returns I'd had recently from being too conservatively invested. That was two years ago and I'm still employed. Since then, the S&P has risen exactly 50% so I clearly would be better off financially if I'd refinanced and invested. Of course, if I'd lost my job or the market had continued to decline, it would have seemed like a brilliant move to pay it off. I could still releverage with a new mortgage but with the higher rates and continued uncertainty about my employment, plus the warm fuzzy feeling I get from not having a mortgage, I'm not interested in doing that.
Ultimately, you will just have to decide how aggressively you want to be leveraged based on your own situation and risk tolerance. And it's OK to admit that emotions and not just numbers play a role in deciding to pay down or pay off a mortgage.
