My DH is an only, and he loved it. His mom remembers him wanting a brother for a few years, but it must have been early b/c DH doesn't remember that. He and his parents get along really well, and money is not an issue at all, thankfully, so we won't have to worry about financially supporting them. We are also lucky that his parents are great friends with my parents, and live within a few hours of each other, so holidays can all be spent together. DH got to do so many amazing things when he was younger that would not have been possible with more than one child on his dad's military salary: skiing in the Alps for a week, travelling all over Europe, going on Outward Bound in high school, etc. He is an introvert, as are both of his parents, but he has lots of friends and the only time he was really lonely was when they were completely new to a place due to a move.
I have a sister who is 26 months younger and we have a relationship that is so awful that it is causing me to seriously contemplate only having one child. She is the complete opposite of me, has no financial or personal stability, no moral code to speak of, and is getting divorced at age 21 while living with her new boyfriend and her baby (by another man completely). While I love my sister because she is my sister, I highly doubt that we will ever be anything beyond civil. I know that I will be the only making decisions and providing support for our parents as they age, and it already irks me to think of that responsbility that should be shared, but will not be.
I never envied only children, though. I never wanted to be an only child. I just wanted a sibling who was actually nice and got along with me and my parents (the 3 of us have an amazing relationship). So I envied the families that had anywhere from 2 - 4 kids, and the kids all got along. As someone said, siblings are a wild card! Well, I suppose kids are in general right?