I'm sorry you've experienced these reactions - I know how frightening they can be!
It's hard to narrow down what you might be reacting to without knowing exactly what you ate or any other factors that might be unique to the times you had the reactions.
FWIW, Benadryl is for comfort only and should not be used to treat a serious allergic reaction. It will not stop nor prevent anapylaxis just as Tylenol may reduce your fever but won't actually help whatever is causing your fever in the first place. I would suggest talking to your doctor about the reactions and possibly getting a script for rescue meds (EpiPen, etc.) and a plan when to use it. Here's a link to the plain English version of an anaphylaxis grading chart
http://the-clarkes.org/stuff/ana.html . (The page has been slow to load for me lately, but I can usually get it with a few tries.) This can help you determine the severity of your reactions. A very basic rule-of-thumb would be if two or more body systems are involved it constitutes an emergency.
ETA: Here's the same chart I linked to above, but in clinical terms (scroll down to Table 2)
http://pediatrics.aappublications.org/content/111/Supplement_3/1601