Please bear with me while I give you a long winded explanation (aren't all of my explanations long winded? :LOL) of the anatomy and what happens.
There is a ring of tissue around the tip of the foreskin called various names including the preputial sphincter, the frenar band, the ridged band and Taylor's band. In an infant and toddler, the purpose of this band is to keep contaminants out of the preputial sac. In young children, this band is not elastic. As the child's body prepares for retraction of the foreskin, the tissue in the band is replaced with elastic tissue and the opening of the foreskin becomes stretchy. At the same time, the epithelium layer that bonds the foreskin to the glans breaks down and is not replaced. When this process is complete, the foreskin will become easily retractile.
When the preputial sphincter is forced to stretch by premature retraction, this non-elastic tussue will get microscopic tears in it and these repairs are replaced by scar tissue. This scar tissue will remain and will not be replaced by the elastic tissue and scar tissue is not very elastic. When there has been considerable trauma such as this, the preputial sphincter will remain non-elastic to some extent depending on the amount of damage done. I suspect that the redness you report is the symptomatic indication of that damage.
From your description, it doesn't sound like you did this a long time and it also doesn't sound like there was substantial damage done compared to other cases I have heard of. There is a chance that your son's foreskin may have lost some of it's elasticity but I doubt it is substantial. There is no reason to approach this issue now. Just wait and see if he becomes naturally retractile as he gets older. When he is around 12 or 14 years old, just ask if he has any problem retracting and if he does, he can use the stretching exercises and all will be well.