The general rule of thumb is 1% but we're good at math, so we don't bother with that.
28 years @ 3.5% no extra payments
$715 payment
$87,229 interest
28 years @ 3.5% making 15 year payment
$1094 payment
$43,879 interest
15 years @ 3.25%
$1075 payment
$40,515 interest
As you can see, faster payoff is what saves you the money. Roughly speaking, if you're deciding between paying $1094 each month with your current mortgage vs refinancing to the 15 year and paying slightly less, you'd save $3365 by refinancing, which is likely more than enough to make up for closing costs.
But...
Many would argue that 3.5% is still cheap money. Does any of your mortgage interest qualify for tax deductions (beyond the standard)? How about you spend the next 15 years investing the "extra" in index funds instead? Assuming an up to average 7% annual return on stock investments (after inflation), while this debt is shrinking (both through regular payments and the power of inflation)... maybe don't tie yourself down to the 15 year payment and the one-time refinance expense. Of course, that's a personal preference, a choice you have to make for yourself. I used to get excited about paying my mortgage off early, and I don't regret going from a 6.5% 30 year down to a 5.375% 15 year... down to a 3.375% 15 year. But now I'm no longer trying to pay off my mortgage as fast as possible. I'd rather make money on investments instead!