I'm an only child, and we're also planning for only one child.
I absolutely LOVED being the only child. I had a friend who had two siblings, and every time I came home from her house, I told my mom "don't have any more kids!" My mom had all the time in the world for me. I didn't have to share her. I didn't have to share my grandparents. I didn't have to share my aunt (who also had no kids). As a result, I had the best life growing up. Trips around the world everywhere, adults who had plenty of time for me, adults who played with me, etc.
I had peers at school - I didn't need them at home!
I still love being an only child. I still don't have to share my mom or grandparents. I get them anytime I want - all the holidays, all the birthdays, and any other time I just want to see them. I never hear "oh, no we're visiting your sister/brother that week."
Anyway, that's my experience. It will depend on the kid. I know some only kids who said they were lonely. But, I definitely wasn't. I loved, loved having my family all to myself.
I personally can't handle parenting more than one. Which is why we're doing the only child bit as well. I'd love a child - but JUST ONE! lol I don't think this is wrong or selfish. I just think it's realistically understanding your boundaries and ability to be happy. Some people can do more. I can't.
Re: loneliness in the distant future. Eh, I don't worry about it. I enjoy spending time alone. And, since I will be having a child, maybe she'll take me in for the holidays when she's an adult.

I plan to be the 80 year old woman who still travels the world everywhere.