I think it is totally appropriate to tell the little boy "You'll have to ask your Mommy and Daddy why yours is different." instead of wrestling with the issue yourself. After all, you did nothing to your son so why should you have to handle those uncomfortable moments. Put it back where it belongs and let them deal with an uncomfortable situation that they made for themselves.
If they have anything to say about it, just tell them "I left my son normal and natural just like he came into this world. It is not my place to explain your actions with his genitals." That should end the discussion right then and there.