Nope, I think you are right.
Build up a good emergency fund (say 6 months expenses) and then 'overpay the mortgage' by offsetting in your mind additional savings in regular savers or high interest paying current accounts (that's probably where your emergency fund belongs as well). After two years you may find that paying a lump sum off (while staying within your 10% limit) would get you a better deal with a lower APR. Otherwise, it doesn't make sense to overpay when you can get higher interest elsewhere.
Or if you have a high attitude to risk, bung in an S&S ISA.
Oh, and don't forget abut your pension!