It sounds to me like you handled it beautifully. Great job, mom, and don't stress about it. Like the PP's have said, this is so common with kids this age.
My oldest son has dark olive skin like his (Palestinian) dad. He went through a similar stage around age 4. He only once expressed dissatisfaction with his skin color, and I told him something similar what you did - that it is sometimes fun to think about what it would be like to look different than what we do, that I've sometimes wished for thicker hair or green eyes like my grandma's, but that I found him absolutely beautiful and perfect just the way he was),
Around the same time, he somehow also became convinced that he was black, and would tell everyone so. I assume because he noticed his skin was darker than any of his other friends at the time, and was working hard on developing his own identity. Skin color is such an easy identifier for little ones to use. Now he is 7, and his concerns have shifted to things other than skin color. He rarely brings it up, and when he does, it isn't emotionally charged.
His younger brother, who has pale white skin and blond hair, is just starting to get interested in skin color now, too, so we're starting to go through it again. He likes to point out how different his brother's skin is. For us, the difference in skin color is such a great learning opportunity about how appearances don't mean much about what is inside - after all, both boys have the same mom and the same dad, and look like they have completely different ethnic backgrounds. It is easy to move from that to the concept that skin color doesn't correlate with any particular character traits, either.