Mind you, I think our cultural (Western European/North American) concepts of manners, etiquette, etc. are deeply rooted in a very offensive class system riddled with sexism and racism. I remember going through an etiquette book once and thinking "wow - 99% of this is about making sure everyone is consistently acknowledging that they know their place". The culture around the concepts has changed, but I think manners should be rooted in the Golden Rule, not in the idea that some people are better than others.
I think the above statement kind of nails for me my feelings of not feeling comfortable with some of these terms. I also live in the Northwest, where sir, ma'am are rarely heard. I work at a shelter for homeless kids and one of the youth who was black and I believe from Somalia, but who had also lived in the States, would call me Miss Lesley. This was the first time I had ever been called this. It really made me feel uncomfortable. I knew he was being polite and showing respect, but I finally talked to him and asked him to please just call me by my first name.