I really, really urge you (OP) to have your kid evaluated by a developmental optometrist who is certified by COVD -- www.covd.org. Discussed here.
Check out this thread, which I started one year ago to figure out why my super-smart little girl, then 5.5, could not read even though she seemed to really want to. I got great advice, particularly from posters laundrycrisis & lauraloo, to have DD evaluated by a certified developmental optometrist. Lauraloo (at post #31 on my thread) directed me to www.childrensvision.com. Take a look at the paragraph of text on the front page of their site. That's what my kid was seeing when she tried to read. From what you recount, your son is seeing something similar. This could well be due to convergence & tracking issues that are not normally identified in routine vision screening.
There is controversy about vision therapy esp. among opthamologists, sort of like there is skepticism about midwifery among some ob-gyns. But there seems to be near unanimity among those who have worked with COVD-certified developmental optometrists who do vision therapy for kids with convergence, tracking & eccentric fixation issues (all of which my daughter has) that vision therapy is remarkably effective.
My kid has been seeing a COVD-certified developmental optometrist for 10 months. Previously, under the guidance of a highly regarded opthamologist, she used an eye patch for 4-6 hours per day in school for two years, and made modest gains in acuity in her weak (amblyopic) eye. She has made greater gains in acuity alone the past 10 months with NO patching except during vision therapy sessions -- 45 minutes weekly in office, 10 minutes nightly at home. On top of that, her eyes are being trained to work together; the lines of letters no longer jerk & move, and she now reads, light years ahead of where she was a year ago.
Both Lauraloo and our developmental optometrist recommended this book by Susan Barry, a neuroscientist who herself gained binocular vision in her 40s through vision therapy, which addresses the history of vision therapy and how it came to be groundlessly dismissed by many opthamologists:
Fixing My Gaze: A Scientist's Journey Into Seeing in Three Dimensions
Barry, a neuroscientist at Mount Holyoke College, was born with her eyes crossed and literally couldn't see in all three dimensions. Barry underwent several surgeries as a child, but it wasn't until she was in college that she realized she wasn't seeing in 3-D. The medical profession has believed that the visual center of the brain can't rewire itself after a critical cutoff point in a child's development, but in her 40s, with the help of optometric vision therapy, Barry showed that previously neglected neurons could be nudged back into action. The author tells a poignant story of her gradual discovery of the shapes in flowers in a vase, snowflakes falling, even the folds in coats hanging on a peg.
In short ... your kid sounds remarkable. His vision issues sound like they are certainly an issue for him, though they may have had nothing to do with the gifted program screening process, which I agree with previous posters you should read nothing into -- says more about the screening approach than about your kid. But the vision issues -- definitely something you don't want to let go.
Good luck! Keep us posted!!