From my experience (which is also borne out by statistics) it isn't the strangers about which you has to worry, but the people you know. Friends, family members, neighbors are all far
more likely to abuse a child than a stranger. I was sexually abused by a family member. Several of my girlfriends have indicated the same (or a neighbor, or other). I only know of one person who has had a sexual abuse encounter where she didn't know the person (stranger rape). Everyone
else knew their attacker and none of them were suspected to be "bad people." While the girls are more likely to be abused, there are an amazing amount of boys who also are sexually abused. One of my mother's best friends had to listen to her teenaged son recount to his therapist about how his grandmother had sexually abused him. My SIL knows her brother was abused by their stepmother. The list is (too) frightfully long.
Overall, I have to agree that women seem to be much more invasive of my child's space than do men. Some of them just downright give me the creeps!