OK, a few more details, none of which really sheds any light on the subject for me, but maybe they will ring bells for other people.
He recently changed preschools (about a month ago). The new school has more kids per class, but also more adults. The beginning and end of day routines should be easier on him (fewer room changes, consistant staff). However, preschool doesn't bring out the strongest response in him. Kids in his class are exactly his age, teachers are great and are being extra gentle with him, allowing him to adjust at his pace, opt out of activities, sit quietly by himself. End of day reports are always positive.
He got a little sister 3 months ago, but seems to have adjusted really well to this. He's great with her!
It doesn't seem to matter if there are adults or kids in the room -- he's still really bothered. Even in a big place like the grocery store (not at "rush hour") when people aren't paying particular attention to him.
Its worst at church, but no changes there recently. He does get lots of attention because we're an active family and he is the unofficial "choir mascot" -- spent most of his infancy in a carrier on my chest at choir rehearsals, so everyone thinks he is "theirs".