I'd not worry :) He'll get it. My almost 11 yo isn't reading very much yet, but his brother was the same way. We tried lots of things with his older brother, but nothing seemed to click. We listened to tons of audio books--think 6-10 or more young adult/adult books a week!, read to him, he saw us reading, etc.
My parents were hysterical that he was 9 and couldn't read so they offered to pay for 6 mos at Sylvan (at nearly $400/mo GASP) because they were sure it was 'my' teaching methods. We had him tested to begin and he was at a K6mos level if I remember correctly, but his vocabulary and comprehension were that of a freshman in college (hello audio books!). They said he OBVIOUSLY had some undiagnosed learning disability and they'd get him 2 grades at least in those 6 mos. So, we took him.....for 4 mos as he lost interest but we stuck with it. They gave tickets or something where you could earn a treat/toy and that kept him intersted for a while. So, they did an exit test and he hadn't advanced even 1 month in those SIX months....so much for 2 grades. They of course said it wasn't THEIR methods...he needed officially tested by a professional for a learning disability!
So, we kept up with what we were doing. My parents were less fussy as obviously it wasn't just me who couldn't 'teach' him, and at close to 11, he decided he really, really wanted to read Harry Potter. He sat down with Hooked on Phonics Master Reader on the computer and finished grades 2 to 6th in about 3 weeks. And then picked up books and began to read. Hasn't stopped since. He's a voracious reader and has a book with him at all times.
All through the times he couldn't read well....he still always felt he was a great reader...because no one told him differently. Now, at almost 15, he's a better and much more prolific reader than anyone his age we know. And has an amazing vocabulary and sarcastic wit!
So, I expect here shortly, his brother is going to spring through the same steps he did as I can see the sparks there with him as well. He's a much more physical child and always moving, running, playing, digging in the dirt, building with his hands, and the thought of stopping and spending hours to learn something just hasn't occured to him yet. He knows sight words for things he uses on a regular basis and we read for him if he asks us to. He'll get it eventually!