I'm going to follow GuildJen and say something that i hope will come off as gentle and caring, but i was also struck by your last post.
Being comfortable having your dd with other people to the extent that you would let them change her diaper isn't something that - imo- should be a 'goal'. you have alluded in your posts that you have history and issues that you are working with that influence your comfort level with dd and men as well as strangers and personally i think *that's fine*. If your dd grows up a little more protected.. well, just be honest with her about it as she gets older and asks questions as it comes up.
Your situation with your neighbor sounds great... but, i dont' know. If it was me, the first folks who took my dd away from me besides family wouldn't be taking her for 2 hours. I don't care if it's across the street. You also said you "knew" she wouldnt' be alone with the men. How could you *possibly* know that? I mean, it doesnt' even matter *who* she would be alone with. I dont' know, i guess it seems like you have this idea that if you felt comfortable leaving your dd in x.y.z situation (whatever you have decided in your mind is 'reasonable') that = some resolution on your own issues.
I feel like i'm not explaining myself right.
I have been lucky - thus far- to have had 99.9% positive experiences with people (won't single out men here) in my life. No assaults. Nearly nothing.
But- I am totally conscious of who changes my dc's diapers or who they spend time with without me. Neither of her grandfather's offers to change diapers, or will, if they are in their care. They pass them off to me or their father or grandmother. I would never have handed off one of my dc's to a neighbor who i have a new relationship with for 2 hours. Maybe 15 min. It would take me several visits, my house and theirs, and some shorter dc-only visits (like 15, then 30 or so...) before i'd get to that chunk of time. Not that time makes any difference... anything can happen in 5 min. let alone 2 hrs. but... testing the waters, yk? Did they have fun? What did they do? How did i feel about the situation? etc...
I just think that if you are a more cautious parent- GREAT. Don't set yourself and your dd up to some 'standard' that you think signals healthy relationships with others. Go by your *own* gut and what you know to be true.
I think your process and decision to remove your dd from your G's care was courageous and difficult... way to be a mama-bear! Your dd is young, you'll get lots more practice to come!

(hopefully in situations not as heartwrenching as this one!)