Wait, your DH was abused as a child, right? And you know for a fact that other parents there use belts on their children?
It doesn't sound to me like your panic is for nothing. It sounds like you know you're about to take your wee little babe into a veritable nest of known child abusers. I'd be freaking out too!
Honestly? The baby is still so young that it doesn't matter who wants to hold him. Wear him in a sling and just nurse nurse nurse (assuming you're BFing). Don't feel the need to pass him around if you don't feel comfortable with the person who wants to hold him. You're the mama and you get to decide whether he will be held or not.
But seriously, wear him in a sling. He'll probably sleep most of the time and few people have the gall to reach into a sling to manhandle a baby.
If I were in your shoes I'd get very comfortable using lines like, "Thank you, but he's very comfortable in his sling right now" and "No, you can't hold him but let's get a picture together!"
I would definitely NOT let the baby out of my sight, given your DH's family's history. Yikes.