We love Sweet Sue's on Main Street in Phoenicia for breakfast. Phoenicia is a great little town, it's fun to walk along the main drag and the Esopus creek is right there.
If you're train fans there's a little train museum in the old train station in Phoenicia and train rides along the Esopus with stops there and in Mount Pleasant.
I also lovehttp://www.craftspeople.us/
It has tons of hand crafted items by lots of different artists.