What are the investment options in the 401K? Are you in a high or low tax bracket?
A 401K usually has to be very bad to justify skipping it. Even if you're not using a 401K, the math says keep your low, fixed rate mortgage as long as possible and invest as much as you can in higher yielding investments.
That being said, I'm personally paying off our mortgage early. However, we're not doing that until all of our tax advantaged accounts (Solo 401K, 457, 2 Roth IRAs) are maxed for the year. We also are fully aware this is a sub-optimal decision from a mathematical perspective, and are doing it anyway for peace of mind, particularly with my wife's upcoming career change possibly meaning we'll be a 1 income household for a couple years in the near future.