Sorta off-topic, but my really good buddy has an American Bulldog, which is squattier and heavier than your standard pit bull.
Anyway, I was visiting him in April with my 3 year old and one year old. Moira (the dog) kept barking at the baby every time she was on the floor, so I made sure she wasn't near the dog when I put her down. She didn't make any rushes at the baby, but the barking (accompanied by a puzzled look) made me very, very nervous.
My buddy got really offended, "I don't think she'll bite the baby!" and other comments. I finally hauled off and told him, "Dude, I don't CARE if I'm hurting your feelings, she's a bulldog and if she bites Sarah, it'll be the LAST thing she does!" He got really huffy but made no more comments.
He apparently thought that I was being "paranoid" about his dog "maybe" biting my daughter. I wasn't about to take the chance of a 40 pound bulldog up against an 18 pound baby.
This is a dog that when she makes a mistake and you swat her, she completely ignores it. He's even admitted that when she gets into something she shouldn't (say, trash or a sock she's chewing on), he has slapped her hard enough to make his hand sting for a half hour and she completely ignores him.
He's resorted to twisting her lip to get her attention.

: So even he knew that if Moira decided to take a chunk out of Sarah, we'd have to club her over the head with a tire iron to get her to let go.

:
So I think you did the right thing. I've had dogs all my life and they are unpredictable. Even Labs, which I own now. They're dogs, they're not people, and you have to get over yourself if people are worried they might hurt their kids...because they might!