I know some of the posts about retraction (even gentle) here can be fairly strongly worded, but I do not think it is necessary to have such a fear of moving one's son's foreskin. If you did move it and it moved easily, no big deal; sometimes this will even happen accidently just wiping up with a diaper change; again, no big deal.
The point is, is that YOU don't need to be the one moving it around to clean under it, check on it, or do anything special with it. By the time your son is 4-5 years old, he should be developmentally ready to learn to take on an occasional "Retract/Rinse/Replace" in clean tub water or the shower. Until then, all that needs to be done is continue to wash off the outside, or just soak him in clean bath water, just as you have been so far.
The tip of the foreskin is typically pinkish, because of the vascular blush of the inner mucus membrane surface, also the glans is normally quite reddish in the intact penis. Neither of these is an indicator of infection in themselves, unless there are significant other symptoms like swelling or pain that doesn't go away after a day or so (these symptoms can come and go transitorily as the foreskin undergoes gradual separation from the glans). So don't worry about that.
Don't feel bad about this. Raising intact sons in a circumcising culture like the US is a learning process for all of us. Heck, I had my boys pre-Internet (1980s) and I was "checking" their foreskins all the time (no other information available at the time), with no harm done. But now I know there's just no need for it, could have potentially introduced bacteria, etc. When you hear an emphasis on not retracting here, it's out of a knowledge that the foreskin is going to take care of itself for the most part, that there is no need for it to be invasively handled (by physician or parent), and to take care to avoid putting too much force on the foreskin before its ready to open up easily on its own. But no need for paranoia!