I think it is all about expectations, my brother and I do that all the time. But we are used to it and pay each other back promptly. If that was not "expected" by you, he should have given you the heads up.
At the end of the day you need to ask yourself what you want. If it is to not have that happen again, either talk to him about it, or bring cash. Yelling at him and causing a rift in your relationship with your brother sounds a little counter productive. Either way, cheers!