I've wondered about that too since DH is allergic to bananas. I've heard that specific food allergies don't usually run in families, but since that's the only one either of us has, I'm going to try to stay away from them for a while. It's a pain not eating bananas right now while I'm pregnant, since I've always eaten them before, but people with banana allergies can also have latex allergies, and that scares me.
Are you allergic to any of the foods with the same protein as bananas? I know kiwis and pistachios fit this category, and others may too. DH is definitely allergic to pistachios, but he's never had enough kiwi to be sure, and I don't let him eat them.
I'm not due with my first until February, but I think I'm going to try to wait at least a year before I introduce bananas. (I'm also going to abstain from them while I nurse.) It will be a pain, but having a DH allergic to bananas is a drag too. Bananas aren't as present as dairy products, peanuts, soy, etc., but they are in a lot of yummy deserts.