CookieMonsterMommy, I'm so sorry about your grandfather!
I was 4 when my best friend's big sister was killed in a car accident. I hope my experience (which I remember vividly) can be helpful to you: My parents explained that S got hurt very very badly so that her body could not work anymore, so her heart stopped beating, and she died. Being dead means she is gone and we can't be with her anymore. That's very sad, and it will take us a long time to get used to it. It's okay to cry. It's okay to feel mad that you don't get to see her again. Let's talk about some of the nice things about S that we'll miss.
I did not go to the funeral, but my parents did. (I don't recall being offered the option. It might be wise to talk about what happens at a funeral and whether your son wants to go. He might appreciate the sense of official commemoration and closure.) Afterward, I asked them to tell me all about it: Did they see S's body; did she look hurt; why not? How was it that they could see S but she wasn't really there? (My Unitarian parents' explanation was something like, "When a person dies, her spirit leaves her body. The body is still real, but there's nobody in there. We don't know what happens to the spirit; it's a big mystery! People believe various things about that, but nobody knows for sure until they die themselves.") What did the minister say? Did my friend cry? Did her parents cry? Did they hug her? (It was important to me to know that she was okay.) My parents were very patient with all these questions.
My great-grandparents died a few years later, and I remember the explanation of their deaths as being more like, "He was so old that his body was just wearing out. That's why he got so sick in the last few years. Finally, last night while he was sleeping, his heart just slowed down until it stopped beating and he died." That didn't sound so bad. I recommend saying something like that rather than talking about how he was sick.
Good luck! I'll be thinking of you.