As far as how frequently to refinance: it depends on your loan balance.
In HCOL's, it's not uncommon to see loan balances of $1,000,000 or more. With that balance, paying one month's payment in closing costs to decrease your annual interest by $25,000 (i.e. one quarter of a point) is a no-brainer. Based on your mortgage balance, it doesn't seem as though you're in a HCOL.
One rule of thumb is to go ahead and refinance if you make your closing costs back in 12-18 months. Knowledge of your own plans for moving, etc., might be a reason to violate that rule.
In my own case, contemplating refinancing a loan at 4.75% in 2012 made me realize that our family just needed to move, instead, which was a much more ambitious (and beneficial) project.