Thanks for all the input. It has made me evaluate what my concerns are with each individual family a little bit. I recognize that most of this is
my issue, but I'm glad I'm not alone.

I got a text yesterday asking her to come over to play and that one probably would have been OK because I've met both parents and the siblings on several occasions. I think I would be pretty comfortable with that family because they all have a good "vibe" but I didn't see the text until too late.
One mom sent a note home with Lexi and I've never actually had a conversation with her because her English is rough. They all speak Hebrew & French, but their dd who is in my dd's class has picked up English. I will definitely have to get to know them better before I will feel comfortable with that one. I just don't know them at all and I'm afraid Lexi won't understand anything they say to her (because I barely understand her in casual hallway talk).
One other mother is single and really admitted that she wants time away from her daughter (while I don't really need that), so I will call her and offer to keep her dd sometime. She actually doesn't go to that school anymore, so dd will be excited to see her.
The other one is tricky and sort of a long story. Her ds has behavioral difficulties and I've talked to my dd about how to model the correct behavior for him instead of avoiding him like the other kids tend to do. He has sort of latched on to her, and his mom really loves my dd. I'm not really sure I can handle him though.

I don't know. I'm going on a field trip with the class next week, so I guess we'll see how that goes before I invite him over.