I've found it really valuable. Almost no one who's come top-rated on Angie's List has disappointed me. The people I've hired who had no reviews, or whose reviews were mixed, have been variable — some absolutely awful. Saving myself from the absolutely awful has been well worth it.
Perhaps the most valuable thing was when I got my roofing done and the roofing company caused a leak into my bedroom ceiling. The company had come top-rated so I was surprised. I called and left messages time and again but no one ever got back to me. So I posted a review on Angie's List with the lowest possible rating and a full explanation of what had happened. At that point the roofing company called me up and apologized profusely and rushed over and made it right (although I had already paid someone else to do some of it, as I couldn't let the leaking go). They offered me some money in compensation, and said they hoped I'd modify my review. So I reranked them as about 3 stars out of 5 or whatever and added the whole story of the leak and the repairs to the review. They ended up giving me about a $400 discount. Anyway, the point of the story is that for some companies, the pressure of a threatened poor review on Angie's List can be significantly motivating.
(I found out long afterwards that the reason the company had lapsed from its normal good service was that one of the co-owners had fallen off a roof and was in the hospital with serious injuries. This explained some of the chaos, but if they had only told me that, I wouldn't have worried about their sluggish response so much.)