I drive and have owned several used Toyotas, and high mileage doesn't bother me much. Currently have a Land Cruiser with 276k miles and a Tacoma with 216k miles that I'd drive across the country tomorrow with no qualms...
A Used one with 100k miles for $20k lost $15k of value over that first 100k miles. At 200k it'll be worth ~$6-8k, so the second 100k miles will cost a bit less but not much. Add in a little maintenance in the 100-200k range and it's a break even with new. The best deal is a well cared for 200k mile Rav4, but that may be outside of your comfort zone. Also, with high mileage Toyotas, care and maintenance vary wildly. I've seen 200k mile vehicles look and feel almost new, and I've seen 100k mile vehicles absolutely trashed, so shop based on care and condition.
Another note, the Rav4 and CRV are wildly popular, and hold their value ridiculously. The last time we looked at buying one, we bought a used Acura RDX (basically a luxury CRV) for less than comparable CRV/RAV4s, so also consider the Acura RDX and Lexus NX and RX.
I'm currently looking for a newish used Civic Hatchback, and have the same issue.