I have told everybody that I will always be responsible for my and my DD's dinner when we are out. I make a point of eating before I go so I don't either go hungry (I am NOT Sweet when I am hungry!) nor do I eat what I know I shouldn't. And I always bring dinner for her, or feed her before we go. I also always have a special treat that she can eat when everyone is diving into dessert. No one around me is like the people you described. I would be livid if anybody handed my DD cheese when I told them she is allergic! Luckily, she is old enough now that she knows what to turn down.
Some folks are sooooo sure that it's "just" lactose intolerance. No, her throat closes up and she vomits. Even so, some "just" intolerant kids have excruciating pain from dairy (so, what the hell?????!!!!!!!!!). Even with lesser allergies I would still be pissed if they were offered to her on the sly. We spread those out so that she doesn't get into an "allergic fog" as I call it, being well familiar with it in myself, of being so overwhelmed by not-so-severe allergies that your body feels sluggish and awful and you don't even recognize individual symptoms any more. I gained 10 pounds after I stopped eating The Big Ones, even though I was eating less! Imagine that my gut was so overwhelmed with crud that I wasn't even absorbing CALORIES let alone nutrients. How this would affect a child, I can only imagine.
I feel so frustrated for you. I hope that people will finally respect your observations and assessments as the person who knows her best.