If he *chooses* to pay when you say you aren't interested, that's on him. You have to let go of trying to control that. You control your choices, which means saying you don't want to buy dinner for 25 people (or an expensive gift, or whatever). What he does after that has nothing to do with you. If he calls you cheap, "that's very unkind" can be a fairly non-confrontational response to start with. But don't try to stop him from buying the gift or dinner or whatever on his own. He gets to make that choice, just like you get to choose not to participate.
This is exactly right. I would add that in addition to saying that it's unkind to call you cheap, I would tell him it's unacceptable. He doesn't know your finances, and unless it's something you could afford but choose not to, it's a low blow. Actually, it's a low blow either way, but I know that most people are understanding if someone truly can't afford something. But they get pissy if they think you are holding out on them.
As for the birthday plans, it sounds like your brother feels the need for the celebration to be special and memorable in a manner that makes sense to him. But he backed down when you and your sister were unwilling to help pay for the restaurant dinner. You have different ideas as to what makes something special, and that's okay. In this case he didn't offer to pay for the entire thing. His dig about "what the majority wanted" says more about him than it does about anyone else.
Going forward, I think an honest conversation is in order. Explain to your siblings that you don't want to participate in joint gifts, period. Tell your brother you don't value eating at expensive restaurants and don't want to participate anymore. Basically let him know that you aren't going to let him spend your money for you.
We are the better off couple in our families. I've learned to deflect expensive ideas with honesty, while remaining as kind as possible. When invited to join several siblings on a planned trip to fly to Michigan to see Green Bay play football, I responded that we appreciated the invitation but that it didn't align with our financial goals. I didn't say we couldn't afford it because it wouldn't have been true. I also didn't say that I think football is stupid. I didn't point out that the people who were planning to go were all deeply in debt and planning to put the trip on credit cards. And when the trip fell through because they couldn't afford it, I didn't say a word.
I don't know how old you are. The older I get, the less bullshit I'm willing to put up with from siblings. On the flip side, I find myself emboldened to be honest with them. I just told a sibling that I'm concerned about gathering at Christmas and I'm not sure we'll be able to make it, or if we do, we'll remain masked. I've finally bowed out of family gifts and gift exchanges, expensive meals out, and family trips (that mostly don't materialize). It took time to get to this place, and I wish I'd had the guts to stand up for myself 30+ years ago instead of always worrying that I was disappointing them.
Also, if we go to a restaurant with family we expect to pick up the bill. I'd rather host people at my house (pre-covid anyway), but sometimes it doesn't work for everyone or they just don't want to do it. If it's just family members we always pay. The others make motions like they are going to contribute, but they always back off and at this point I think they know we'll pay. And we don't choose crazy expensive restaurants. I think the last meal we covered was at an Indian restaurant, with seven of us present. Some had the buffet, some ordered off the menu, everyone one had some sort of beverage, and I the bill was under $200 with a generous tip. Not small change, but we could afford to do it. Would I do it every month? No. But neither would I ask others to chip in and them call them cheap if they didn't.
(Also, we have one sibling who always orders whatever their heart desires -- expensive entree, appetizers, dessert, and a special beverage -- and in the past
never had the money to cover their portion of the bill. They still don't, but now I don't get upset because I go into it knowing we'll pay. But there was a time when it was a hardship because we were all trying to make ends meet and this sibling just couldn't do the math in their head, I guess.)