Going back to my original point, as long as they are behaving appropriately, why not? I think it's far more rational to set standards for people's behavior than it is to exclude people based on age.
In light of that, I can't imagine taking my kids to a B&B when they were young. Not because they were bad, but because they would run and stomp while playing, make incredible sound effects with their Hot Wheels, have little control over their "inside voices," etc.
Now, maybe other people have children that are mouselike, tip-toeing around and whispering, although that's highly unlikely to be the norm. IME, a lot of parents are so used to the clamour that their kids create that they tune it out, and don't realize that others might not appreciate it.