I was very strict with DS the first year +. Breast milk, and when he experimented with solids it was only whole, organic foods. He did have some puffs, but I bought the best I could find with the most wholesome ingredients.
DS is now a bit over 2, and I am more relaxed. Nurses on demand, has a juice box when running errands, but likes mostly water. Treats are allowed, but he knows what is a treat. He has a piece of good quality dark chocolate here and there when I do. I figure if he is going to have a treat, make it a good one. Ice cream on occasion, at the ice cream parlor or here at home. He loves goldfish. Is in love with french fries (who could blame him?). To resolve his (and my) french fry craving, we make them at home or once in a blue moon will get a small one from a drive thru...last time we did, DP was in the car with us and said, "Honey, I feel like such an awful parent right now." I reassured him that allowing him some french fries 2 or 3 times a year from the drive thru will not kill him. It is funny, because I have ALWAYS been the strict one...I must be rubbing off on him!
As a general rule (at home), we eat whole, mostly organic, from scratch food. Very few processed/boxed foods (tortilla chips, goldfish, organic honey sticks, pasta, kashi crackers, etc), no HFCS, MSG, artificial colors/flavors, etc. At family gatherings/parties, I normally don't restrict too much, but even then my DS will choose all of the fruits and veggies over the junky options.
I think teaching our children moderation is important. Teaching them that real whole foods will nourish our bodies when eaten the majority of the time. Treats are just that...treats to have on special occasions.