I think it can be because they have some underlying insecurities about their own decision.
I haven't had a tv since I was 18. Most people don't engage me in debate about it, but those who do invariably feel defensive about their own viewing. They insist they only watch educational tv, for example. I have never had anyone try to convince me it would be good for me to let my son watch it.
I'm a teacher and my students are always aghast when they first find out I don't have/want a tv. They always wonder what on earth I do with my time
I have converted a few of them, though.