I purchased travel insurance for the first time when I was booking a week in Hawaii for our (at that time) family of 3. As the entire trip was a few thousand, the $90 in travel insurance didn't seem to be a bad deal.
It paid off. We missed our connecting flight - due to weather. Which means, we were stuck overnight in Los Angeles. Because it was weather, the airline would do nothing for us and our 5 year old (we did try). The travel insurance bought us dinner and breakfast, and paid for the hotel.
It also refunded us for the missed day in Hawaii - about $400 or so, total, refund for hotel/car. Of course they gave us each a check (including the 5 year old).
So now, we buy travel insurance for:
1. Any expensive trip (honestly this Hawaii trip was 5 years ago, and it was our last expensive trip)
2. Any trip with the kids where the possibility for getting stuck might be high. Usually we fly to visit family, so we don't really feel like we *need* travel insurance. Example: red-eyes. However, this summer part of our trip was a leg with just my husband and our older son flying home, late in the day, with a very short connection in San Francisco. In the summer. So he paid travel insurance, figuring it was cheaper than a night at a hotel in San Fran.