If the main thing that's bothering you is that you "missed the boat", you could also do a one-time additional payment to your mortgage (enough to "recover" the amount you "lost"), then invest the rest.
For example, an additional $25,000 payment at the beginning of a 30 year, $300,000 loan at 4.25% ($1478/mo P&I) would save $55,000+ in interest over the life of the loan, and cut 4.5 years of payments off the end of the loan.
Vertex42 has a really awesome Excel debt reduction calculator that I use whenever I'm daydreaming about paying off my mortgage. You could play around with that to see the effect of additional payments to your mortgage payoff schedule. I like their calculator in particular because you can input either additional monthly payments, or one-time payments at different points over the life of the loan.
Good luck!