I was like this as a child. My sister and I both tested "gifted" on every test available. She could read well before she was 3, and I didn't show ANY interest in reading until the summer before 3rd grade! By the end of the summer I had read all of the Little House on the Prairie series (including the later, more advanced books) to give you an idea of how quickly I progressed once I actually showed any interest. And I now have a MA in Library Science
I think the most important thing is for you to just keep reading out loud to him, whenever possible. Expose him to books, but don't push. Kids like this tend to sink in their heels when the very fine line between encouragement and pushing is breached. And it's very easy for them to see encouragement as pushing. Eventually something will catch his eye, and he'll decide that he simply MUST read it, and he'll sit down, and he'll get it. That's the trick: he really has to want to read something, and decide that he can't wait until bedtime, or that he can read faster to himself than you can out loud.
Take him to the library once a week and let him pick out 2 books a week Really stress the "let." It might go against all of your parenting instincts (assuming you normally allow him to take out anything he wants), but to push him to really examine books, stress how he's allowed to take out two books and only two books, and what a great privilege this is. I mean, don't go into overkill or he'll see that you're trying to manipulate him, but just be firm: "You may pick out two books to take home. Oh, you've found three? Can you decide which one you want to leave here, and we'll get it next week?" Give this a shot for a few weeks, and see if he really starts carefully examining the books: the pictures on the cover, and pictures inside, the length, if he recognizes any words. This doesn't work for all kids, but for some kids it really does allow them to carefully consider each book they handle until they find one that really sparks their interest.
Something else to keep in mind: my parents always suspected that part of my problem was that I just didn't "get" phonics. By the time I was in school, whole word instruction was completely discredited, and my parents were very knowledgeable about educational trends and fully supported the research that a phonics based curriculum was best. But I just DIDN'T GET IT. For years. Finally they started to do some whole word drills with me, and I "got" that. I think that most schools find a balance between phonics and whole word instruction these days, because it's obvious that while most kids learn better with phonics, some are like me and need the whole word immersion.