I think living in a foreign country is a lot harder than most people realize. Everyone's day to day life is full of small cultural literacies, most of which we aren't even aware of. We simply know how to do things the way we have always seen them done, without really even being aware we are relying on a sort of tribal mind. Having those cultural literacies ripped out from under you can be damn hard. I've only lived in places where I didn't speak the language, which no doubt made it more difficult, but most often is it not the language that has cause major frustrations so much as it is the lack of understanding about how things are done.
When I first arrived in Germany, a friend told me this story and I think of it often. She'd only been here a couple weeks and she made her first major Germany grocery store shopping trip. She managed to do a lot of things right which other people screw up (like weighing her own produce and printing the label, rather than bring it up and expecting the cashier to do that, something very different than the way it is done in the States and which most people wouldn't even think to question unless someone mentioned it ahead of time). She loaded her stuff on the conveyor, aware that she was expected to have her own bags and to very quikly bag her own products as they were rung up. She was doing it all right! And suddenly the elderly lady behind her started screaming at her. She couldn't understand, and was soon crying. Eventually, she realized the woman was telling her that she was doing it wrong. Bottles are to be laid sideways on the conveyor. And Germans tend not to be gentle about corrections. They aren't being unpleasant. They are just correcting a wrong and showing you how to do it right. But the manner in which it is done can feel stern and reprimanding.
Many Americans here also have a shared humiliation about their first time leaving a parking garage. You always pay before you leave at a small vending-syle machine, not at the actual exit gate. But if you don't know that, you pull up to the gate and can't get out, and if there are cars behind you, they get annoyed, and you can't back out, and...
These are just illustrations of the many, many small frustrations of Ex-pat life. People think of the big things, but to me, it is the small cultural illiteracies that make for the Bad Days.
That said, I've had wonderful experiences and DH and I have recently discussed the possibility of a semi-retirement gig running a small B&B in Scotland or maybe Ireland. We've done almost no research so I don't have any idea of the logistics, but it does show that even with the headaches and humiliations, I've found much to like in overseas living, too.