Well, I knew deep down, and thought about it at 16 weeks when my fundal height was at my belly button, but then repressed all thought of twins by 20 weeks when my fundal height measured right on (the rest of the pregnancy, even) and I didn't gain any more weight than with my two former pregnancies.
I honestly didn't think about having two, but yet I couldn't accept the fact that I was having either a boy, OR a girl...I couldn't let go of either one of those images, and I had guessed my first two babies correctly (singletons.) And when I felt the limbs (two many legs or arms) I had a slight twinge of wondering, but never anything concrete.
It was my third pregnancy, so I chalked it up to that. Until the midwife felt the lump in my belly after my dd was born, then I KNEW that I had always known, and I felt "validated" in a way...though I was glad we didn't know for the stress it would have caused!