Is there a reason you don't consider something like a Toyota Matrix?
With only one car seat left, you can fit five in that with a decent trunk space.
That seems like it could be really tight with a car seat. We have a Rav4 currently (no kids) but our friends also had a Rav4 (maybe 2007 or 2008) and once they had their second kid (2 car seats) they decided to sell it and get a Honda Odyssey. This is just a frame of reference... not sure what the space is in a Matrix vs Rav4 (especially an older Rav4). I imagine once you have two car seats in there it gets crowded regardless though. But even if you had one car seat and had to get two more in the back, that still seems like it would be somewhat cozy in a Rav4 at least. I'd imagine much more so in a Matrix.
I've got a Matrix and two kids, two car seats. We had a couple of years of having the baby/ toddler seat in the middle, and the boosters on the side (because I carpooled with another family, I had 3 kids in the back and one kid in the front in a car seat. But the carpooling was only home from school, so a short distance).
Even so, my kids are now 2 and 9, and the 9 year old is still short and still in a booster with a back, and there is plenty of space. No worse than a RAV4. (However, three carseats probably would not fit terribly well.) I think the older RAV4s are smaller.
As far as when they get older, I dunno. When I was a kid we went on long car trips to visit my older sisters, 2 hours each way, and we had a fiat for some of that time. We never had "big" cars. (Usually 3 kids in the backseat.)
Someone mentioned that the backseat is like a Corolla/Civic (true! Civic is our other car!) But funny thing, the Civic/ Corolla these days is the size of an older Accord/ Camry. So...not that small.
(Don't get me wrong I would LOVE a minivan, LOVE it. But I'm cheap and I simply cannot justify it. If it gets to where it's too hard to take long car trips for vacation I'll just rent a car.)