I wanted to add "The Trumpet of the Swan." We finished that one recently and my kids LOVED it. (They are 5 and 3 as well). And we've already read a few of the already mentioned books -- "My Father's Dragon," "Stuart Little" (another big hit in our house), "Charlie and the Chocolate Factory" (ok, this one had a bit too much dark humor for them, but they enjoyed it nonetheless). "James and the Giant Peach" (same problem, actually, my kids found it scary)
Chapter books to read to my 5 and 3 year olds? - Page 2
I love threads about children's books!
Most of the books below have already been mentioned, but I thought I'd add a little more info to help you narrow down where to start with so many great suggestions.
We read the My Father's Dragon series when my youngest were 2 1/2 and even at that age they were able follow along for a chapter at a time. I think this series is an excellent place to start. A warning, though--in the beginning of the first book, Elmer's mother spanks or whips (something violent) him for bringing the cat in the house--we just edited. These books were written over 50 years ago.
My kids also loved the Magic Treehouse series. I don't think these books are as well-written as some of the others mentioned, but my kids could follow the stories easily and they are somewhat educational. They are very popular.
We read the Little House series when my oldest was 5. My youngest two were 3 at the time and they liked Little House in the Big Woods, but weren't as interested in the other books.
You may consider The BFG by Raold Dahl for your 5 year old. We've read this book five times in the past few years. I think we started when they were 3 and 5. There are giants that eat children in the book, so if your 3 year-old is not intrepid, you may want to wait a bit but I think most 5 year olds could handle it. My husband and I love to read this book out loud because the language is fun and our kids think it is the best book ever. Fantastic Mr. Fox by the same author would also be appropriate for your kids and it is shorter and less scary. My seven year old is currently on a Roald Dahl kick and can't get enough of his dark, silly and very inventive books. Your 5 year old may also enjoy Charlie and the Chocolate Factory. We've tried James and the Giant Peach several times but my kids just can't get past the very depressing beginning. All in all, though, we love Raold Dahl.
The Mercy Watson books by Kate DiCamillo are cute and funny. They are a nice bridge between picture and chapter books.
The Ramona books are funny, too. I think most 5 year olds could relate well to Ramona and her antics.
We are big fans of Cythia Rylant. Mr. Putter and Tabby and Henry and Mudge books have very short chapters and also make a nice bridge from picture to chapter books.
Hope this is helpful.
We have read most of the above as well. Right now both my 7 yr old son and my 5 yr old daughter love the Fairy series by Daisy Meadows. They are all pretty easy to read and relatively short. In addition my son really liked to read and be read to many of the Mother West Wind series and by Thornton W Burgess. The kids grandfather started collecting them when he was a boy and then my husband read them and now my kids. There are many books by TW Burgess. Many are out of print but some are available as reprints in paperback.
Another great animal story is Rascal by Sterling North about a boy growing up during WWI and his pet racoon.