We are fortunate in that our ds (age 7 yo) has no fear of doctors. He hasn't had a shot since he was 7 months old, so he has no associations with doctor=pain.
Do you vaccinate? Did they do something at his last visit that caused him any pain?
If this had been me, I would have just packed him up and left before the doctor even came in. I wouldn't want my child to see me ignoring his terror by staying. ((I'm not being mean to you or any other parent when I say that!!)) If the office visit was for something important, like the child was really sick or injured and needed treatment (stitches), then you might have to march through it.
But, ignoring your child's hysterics that are that severe at a routine exam, to me, sends them a message that their fears are not important to the parent. It's all very well to say "There's nothing to be afraid of!" but to that frightened child, there certainly IS something to be very much afraid of.
As adults, we know there isn't anything in the closet or under the bed to be afraid of (well, I think there isn't, I'm pretty sure there isn't, actually, I never look because, well, um, just because I don't...). Anyway, children can't always express their fears of something because it's just too terrible to even say. Or, they are too young to make it clear to US (dumb adults that don't understand their infant/toddler language). Children EXPECT their parents to make sure they are safe and not hurt. They expect them to make things better when they do get hurt. They do NOT expect their parents to take them someplace to be hurt on purpose, while Mommy/Daddy stand there watching, perhaps even with tears in THEIR eyes. Their little minds can't comprehend this opposite behavior.
Can your child articulate to you what made him so upset? Perhaps you could schedule a visit to the doctor's office that did NOT include going into that exam room first, but into the doctor's office instead, where the doc could simply talk to your son and ask him things NOT related to health or his body. Fun and silly things that interest your son. Perhaps then, the doctor could ask your son if he'd like a tour of the office and see what's there. Show an exam room (have the doc sit on the table and your son on the little stool with wheels!) and let your son look in the doctor's ears and mouth and use the reflex hammer on the doctor's knee.
Any doc that would not want to be understanding and compassionate about easing a child's fears isn't a worthwhile doctor!