People have given a lot of good examples of when PMI is a good idea. I'm going to play devil's advocate a little. You gave an example of PMI being 0.05% of the loan. In this case, it would be $125,000 x .0005=$62.50/month. However, that means that annually, it comes to $750/year. This is still only 0.6%, except it's not really. This is really only interest on the next 10% (or 12%) of your loan, so really, it's 6% of a $11,250 loan that you're already paying around 4% on. Effectively, a 10% rate on the top 10% of your mortgage, the first year. PMI typically lasts for a minimum of 24 months.
If you pay $5,625 in principal the first year, you are still paying $750/year in PMI. For the second year, this means that the effective rate of your PMI is 16% on your remaining $5,625. This means that you've had a two year loan of $11,250 at a rate of 13%. In addition, some lenders require another appraisal to remove PMI before term, no matter what, which tacks an additional $300-ish to that cost.
This leads to a reasonable question: is a 13% loan (plus $300 appraisal fees) worth it? In addition, if you don't pay down principal quickly, the effective interest rate of the PMI goes even higher. It's not as straightforward as it looks.