I'm an only raising an only. DH has a sibling, but they are not close at all. Before DD was born we had already planned to have just one, and now that she's here it feels completely right. I love our family size and I love that we can still manage to do all of the things we did before becoming parents without taking a financial hit. We are on a pretty moderate income and I think our quality of life would go down if we had to pay for another child. I feel shallow saying that, but it's true. I will say that I do find DD to be draining at times because she does seem to demand a lot out of me. But I just assume that that's not unique to her and if I had another one I would feel even more drained. I don't know that I can articulate how right it feels for our family, but I never feel that longing for another.
It's kind of funny because I thought as she got older I might start wanting another one, but that hasn't happened at all. Her getting older is actually having the opposite effect on me. As she gets older I find that I am able to pursue some of my interests that have been on the back burner since mothering took so much of my time.
As other have already said, it really just comes down to what feels right for you, as there are absolutely no guarantees for how things turn out no matter what direction you go.