I don't have twins but just saw this thread title and wandered in. I was friends with a fraternal twin when I was in middle school. If I recall correctly, I think I could tell them apart before I was friends with the one, but I wasn't very sure (as in, "I think that one's Megan" - and I had to look for certain cues to confirm). After I befriended one of them and got to know her, her sister looked more and more different to my eyes until I could instantly tell them apart even from the back, or fairly fair away, etc.
I understand that people differentiate first based on sex, then race close behind that, then hair after that (color and style), then age markers (which is often size in children and skin in adults). Major differentiators such as a major, visible physical disability might come before any of those (i.e. you see a person in a wheelchair in the halls of your workplace, and you know only one person in a wheelchair in your company, then you identify them - possibly mistakenly, of course - without even having to see their sex or race).
Anyway I say all that because age markers are a pretty big one, and twins defy that particular differentiator. I think people "see" features (such as nose shape, lips, presence or absence of freckles, etc.) far less than we would assume, but identify a person on far less nuance than that. Did you ever have someone you sort of knew have a major hair cut (and possibly color) and when you saw them you had to spend an extra beat thinking "Oh, that's Anne" (and in that case you might not have recognized them at all out of context, say if she were a coworker but you saw her at the grocery store as opposed to work)? Therefore showing that you identified her almost entirely based on sex, race, then hair (and context, of course).
Also, I've read (I think in Reader's Digest) very recently that some people actually have a handicap where they can't distinguish people - and that handicap is on a spectrum, so some people can usually get by but still have trouble in certain situations, like twins that look like siblings but are the same age. Since hardly anyone has heard of this handicap I would imagine most people aren't even aware they have it, just find it hard to identify people in everyday situations.
Anyway, I'm sure it must be frustrating, no doubt, to have your very different looking twins confused all the time. But just wanted to throw in some perspective that people outside of intimate family literally do not "see" your kids the way you do, and while some of it is still ridiculous and probably lazy (like when your kids appear to be different races, lol), some of it probably is pretty normal for acquaintences.