I'm allergic to ragweed (now - its only a few years old), and used to drink chamomile tea. Then I learned that chamomile and ragweed are related. I've stopped drinking the tea even though I never had a problem. Better safe than sorry for me. It may also explain why I don't like melon, zucchini, and cucumber - they're also related.
My guess is that it would depend on what specifically is triggering the reaction. There are compounds in penicillin that aren't in blue cheese, and vice versa. However, it is possible to develop an allergy, or worsen an allergy, through repetitive non-reactive exposure. So you might be fine for years until your immune system suddenly decides that you're not.