Former player's option sounds kind of tetchy to me. I think it's important to remain resolutely, even obnoxiously, cheerful. And not to commit to anything this far in advance! I'd go for:
This year: "Thanks for coming over to ours on Christmas day! We really appreciate you making the trip now we have a little one to look after, and I know [son] had a great time playing with you. I look forward to doing the same thing next Christmas when he'll be old enough to get excited about [Santa/tree/whatever] too!"
Then next year you just shift those goalposts. Two meals was fucking insane and I think even one meal is bananas because as marble_faun says you just won't get everyone happily round the table. "Hey, [son] is really excited to see you again this year! We won't be able to manage a meal* but we'll have [mince pies and mulled wine/tea and cake/coffee and pastries] out from [time] until son's [nap/lunch/dinner/bedtime] and it would be great to see you. Let us know what time you think you'll be able to drop in."
*LITERALLY NO MENTION OF WHY EVER
I'd do morning by yourselves, lunch by yourselves, open doors between lunch and your son's dinner, then chuck everyone out and have an evening by yourselves. I am really good at nicely throwing people out of my house and I do it by being totally explicit: "It was so great to see you, but I'm afraid it's time for ToddlerSLTD to have his dinner and go to bed so I'm going to have to chuck you out now." Coupled with physically standing up, smiling, and saying let's text and make plans for next time. No hints, no little nudges. And not a random time, but the implied reason that otherwise ToddlerSLTD won't have any dinner. (Partially true. He certainly won't go to bed happily if fun is happening downstairs.)
