I am LDS, and since no one else has replied to your post, I just wanted to say that often we (Mormon, Latter-day Saints) feel like we _are_ singled out because of our differences. Maybe that's why the LDS people you have met are more accepting of you. We also have a teaching from our "Articles of Faith" by Joseph Smith, which he wrote in explaining some of our beliefs:
"We claim the privilege of worshipping Almighty God according to the dictates of our own conscience, and allow all men the same privilege; let them worship how, where, or what they may."
(This is almost exactly right--a near quote!)