When you do that, you're basically speculating on currency.
If their dollar (or whatever) falls versus ours in the meantime, it could more than wipe out your gains and turn them to losses. If theirs rises versus ours, it could way overshadow the small interest you're getting.
If you have no currency knowledge, then you're basically gambling hoping it stays the same or goes up.
Unless you are living in that country, so your expenses are in their dollars (so the exchange rate doesn't matter to you), it's quite risky.