I'm the same way. No one can tell, unless I'm wearing a tight shirt whilst I'm working out. When I workout, I get BH and my tummy grows massively! If I'm just in regular clothes, no one can tell, and I'm almost finished with 24 weeks! I'm about your height, maybe 30lbs lighter(after a 20 lb weight gain in the first tri - yikes!), so I thought I would show pretty quick, as I was thin, but I still keep hiding the baby! I've popped some, but it just looks like an average-sized tummy now. I think abs play somewhat of a role in it, as I HAD 4-pack abs, which I'll probably never see again. But I also think that the length of your body and space in your torso is the major factor in it all. I have a friend who is a personal trainer, had awesome abs, but popped out pretty quick as she has a shorter torso. I have a long torso, so lots of space for upwards growth, and I'm not forced to grow out, yet.
I have found if I wear tighter clothes, that helps and if I wear maternity clothes, well, they make ANYONE look pg! Like lrgs said, soon enough we'll all be waddling around!