I think this REALLY depends on the kids (or, toddlers, really), that you have. Our 2.5-year-old is insane. He runs, he screams, he doesn't listen, he won't go to sleep, he throws things (i mean, he's also very smart, inquisitive, polite, and can be sweet too - he's like Jekyll and Hyde - and that's what makes it tough, never know if he will say "please may i have some more toast?" or if he will throw his plate and scream and yell until he gets toast). His younger brother, 9 months old, is the happiest baby ever. He smiles at everyone and everything. He isn't mobile. He's content sitting in 1 spot playing with toys for 20+ minutes at a time. His older brother was never like that.
I joke that i want 7 kids with a huge van out in the country on a homestead (isn't that everyone's dream? ;) ). But, honestly if we get another child like our first, it could seal the deal for us. If we keep spitting out children like our youngest, i could see my wife agreeing to go to #4. If we had our children in reverse order, we would be done at 2, almost assuredly. Our first would have lulled us into a "babies are easy!" mentality and then the second would come along and blow our socks off. After our actual first, we expected all our kids to be like that. When my brother had a baby a few months after us we were like, "what's wrong with her? She just lays there all day!!" and now we would kill for another baby like that.
So, i think the personality of children plays a role too.
Also, there is a book called "Selfish Reasons to Have More Kids: Why Being a Great Parent is Less Work and More Fun Than You Think" and I read it, and it was OK. It made some good points, but it also called out the fact that generally speaking, people with more kids are less happy and that taking care of your kids is one of the things that make us least happy (seconded only by driving to work). It did say that PLAYING with kids was high on the list for happiness though, but again, how often are you "playing" vs. "watching" your kids?
One thing it called out as a big consideration is that kids are around for life, and most people don't take this into consideration. They just think, "Oh damn, i already have 2 toddlers, how am I ever going to sleep again if i have another kid???" But you need to remember that the toddler phase is like 2 years, and these kids are going to be around for the rest of your life, not just the first 3-4 years of their lives. It said that grandchildren make people very very happy, and that your likelihood of having grandchildren increases (obviously) with the more children you have. You have potential built-in caretakers/support system as you get old, infirm, or sick.
It also made the argument that you might have a somewhat moral obligation to increase the young population because we are going to have a serious crisis when we are all old and there are no children to pay for social security, take care of us, or keep economy going as the population makes an inverse age pyramid. Interesting thoughts.