I'll put in another vote for reading Quiet - for both introverts (and those you love them!).
I think about small talk in 2 ways/scenarios:
1) I'm just passing by and don't really want to get to know you - these are the polite acknowledgements to the neighbor, the cashier, the person on the bench next to me, and a lot of my co-workers. I smile, I'm polite, I'll engage beyond hellos if they want. But if they start peppering me with questions, my answers start to get real short - I don't share much with strangers. (my neighbors rarely come outside, so yes, they are strangers to me)
2) We're confined in the same area for a while and/or I think you might be interesting (group hikes, parties): here is where I will put in some more effort to be social and see if I connect with someone. If the conversation seems to be strained (list of questions and not a flowing conversation), I'll politely excuse myself for (another drink, bathroom, fresh air, tie my boots). If a conversation feels like work, then I don't invest any more effort in it - accept that we're not connecting and move on.
And for the record, I hate when relative strangers ask me questions like, "how'd you meet your spouse", "what books are you reading" - these are generally random questions that have no context and feel like an interrogation. I'd rather the conversation went, "do you guys have kids?" (common question and they are trying to find common ground), "no, we went the dog route instead" (said with a chuckle)... and if they want to pursue the conversation, they can easily say, "oh, what kinds of dogs do you have?".... then it feels like more of a back and forth conversation where I can tell them a little something about my dogs, ask if they have any pets, and hopefully this leads to other topics if we have a vibe going. It feels like a much more authentic conversation born out of genuine interest about me/them, not a set of questions they really don't care about the answer. Now, if they don't ask me about my dogs (or something else), then I assume they're not really interested in a conversation... and that's cool, too.
The other tactic I might try if I'm trying to see if I might connect with someone, is explicitly ask them if they ever do X, Y, or Z (things I'm interested in). If I get a hit on any one of those, then those are topics I can jump on to and get a good conversation going. I rarely ask someone a question I'm not genuinely interested in the answer, unless I'm just being polite and perfunctory (ex: do you have kids?... for me, the last thing I want to hear about are the tales of your kids, especially if I don't know you/them). Once I get to know you, then I'll probably ask about your kids because I'm interested in you/them. Makes sense?
My MIL is the queen of chatting up everyone and anyone - walks up to complete strangers in the mall and asks them where they bought their pants because they look sensational. Scolds me for not inviting the neighbor in for a drink (neighbor dropped of a mis-delivered mail). She's just wired totally different than me. She's fun to go out with though, because we get to be quite social, with virtually no work on my part!!
My husband is in-between. He'll call up a Big Box store looking for something, "Hi Karl, this is Robert, how are you? Great, hey, I'm looking for ...." and I tell him that no one cares what his name is. And then we go to Big Box store, he finds Karl and re-introduces himself, shakes his hand, "we spoke on the phone". Karl remembers him and somehow now they become pals and start chatting about stuff while Karl finds what my DH wants". DH is just naturally friendly.... and he always gets great service because he's good at forming a genuine connection. But damn, if he can't remember the details of the conversation or people's names!!