I agree it's wise to follow your instincts. I also agree that most kids don't like being poked and prodded by a doctor, although frankly, kids who go for regular WBV and are vaccinated tend to start screaming as soon as they hit the waiting room because they remember what happens there. So going to the doctor on a regular basis doesn't necessarily acclimate a child. In fact, I think it's probably not a bad thing for a child to not want a stranger to poke and prod them. I don't like it either!
All that said, it sounds like this doctor's beside manner is lacking. We also do not vax or do WBV. Besides a few visits just after she was born, I've taken 2yo dd to a doctor twice; once when she was sick (to make sure she didn't have an ear infection), and again a few weeks ago to establish care with a new (and highly recommended doc) so we would have a relationship with someone in case dd needed to go in. Many doctors operate the way you describe -- dictating to patients, making decisions for them, not supporting their wish to educate themselves and make their OWN decisions. But there are doctors out there who aren't like this. The family doc we just started seeing isn't 100% perfect (e.g., she advised weaning 2yo dd if I get pg because she said it's hard on my body to support two children ... which is true, but dd really only nurses once/day, so I'm willing to continue on as long as it works for both of us), but she does listen to me, respects the fact that I have a functioning brain and am not a zombie, and we're talking about MY body, and MY child, and in the end I get to decide.
I think the fact that he didn't share any information with you is deplorable, and an indicator of his style. I would keep looking, and find someone who will listen to you, explain things to you, and support you in your efforts to make the best choices for you and your child. I know he was recommended as supporting AP, which is nice, but it doesn't make him a nice doctor to deal with. If you haven't asked around in the "Finding Your Tribe" section for you area, try to get some recommendations from other MDC mamas who live near you. Specifically state that you're looking for someone who supports AP, non-vaxing, AND will communicate with and respect you.
Good luck. This is not always an easy quest ...