Hi, Slighty Crunchy! (love your name),
I think your son sounds very bright. And I think what you're doing with him is fine. From what I've read, not writing or knowing letter sounds at 3/4 is normal. This thread has a self-selected group (i.e not the norm), because it mostly attracts people who do formal stuff in the preschool years by virtue of its title. I, personally, don't think you should change what feels comfortable for you. I think it has been shown that, of kids of the same ability, they even out down the road, so you're not at a disadvantage for not doing formal stuff. I think you should just follow his cues. When he's receptive to learning it, he'll easily pick it up. My BIL swears that the reason he hates reading so much is because reading instruction was pushed on him in kindy before he was interested in doing it.
My nephew went to kindergarten at almost 6. He could only write his name, didn't write all the letters yet and didn't know most letter sounds. I'm not sure he could count very high. No one had worked with him on academic skills. By the first grade, he had completely caught up with most of the other kids in reading and he was advanced in math. He simply was ready and once he was ready, he took off.
I loved the book, "Einstein Didn't Use Flashcards". It referenced a lot of studies and was very reassuring. A lot of people in this forum recommend that book; it's great! Like did you know that the Mozart Effect simply showed that college students performed better on a very specific sub-test after hearing 15 minutes of Mozart? Yet, it's marketed like classical music will make a *kid* "smarter". The actual study showed very specific and limited results on young adults. We listen to classical music, because we love it. We only do things we love. I can't speak highly enough of that book.
Speaking of which, there's another great book about the love of reading called, "The Read Aloud Handbook". It's by Jim Trelease. He convincingly makes the argument that reading aloud to share the pleasure of reading is significantly more important than teaching the mechanics of reading, if the goal is to make a life-long lover of books. It was also reassuring. I have a child who can sound out small words, but doesn't read. We've been at this plateau for a long time and this book reassured me that it was Ok. It's more important to read to them like you do and to make reading a pleasurable activity, because that creates motivation to do it at a point of readiness.
In terms of reading pleasure: We're not reading great literary works.
My son loves the Magic Tree House books and even though I've received advice to read higher quality stuff to him, this is what he LOVES. It's better to keep reading as a pleasurable activity. I could try reading stuff that's written better, but if it doesn't excite him like the Treehouse books, then I can't see the point. I want him to love reading so we read stuff to him that he loves. Even my very well-read husband went through a very long period of reading nothing but comic books, my MIL said. She never bothered him, because he was enjoying reading rather than being turned off. It was better to read frequently for pleasure rather than drudgingly reading better stuff out of obligation. Now, he's a voracious reader.
I love that your son memorizes the books that he loves. This is very smart. Sometimes kids figure stuff out that way. My sister taught herself to read without any instruction, not even alphabet sounds. My mother just read to us every day and pointed to the words sometimes and my sister was able to figure out the code. On unschooling lists/threads, there's mention of lots of kids learning to read this way. The fact that your son loves being read to is THE most important factor in reading success, or so says Jim Trelease. Again, memorizing favorite books shows that he's very bright and that he loves reading; you're setting up a very strong foundation for him.
I've probably rambled on too much here. But I wanted to comment on what you said and say that I think you're doing a great job. Honestly, my kids (oldest one is almost 5) just play all day long. I basically stay out of their way most of the time, because they're busy playing. When they play, they really stretch their brains to make them powerful machines for future academics. Play is kids' work. We read, we do some art, my oldest does some wood-working with dh (I noticed your dh does this). We just play. It's really important to me that they have an unhurried childhood. It will all come in time. If they learn it at 8, as opposed to 3, they probably learn it so much faster most of the time. I would just follow his cues and do what feels good. HTH!