we've never lived near a wal-mart, but i've done some great simple traditional foods shopping at one near where we vacation.
I second all said above- whole foods, whole grains, veggies, and the best quality meats you can buy. we shopped for 80% of our traditional foods through our local grocery store- and our shopping style and community shifted the store's inventory. they carried raw cheese, pastured seasonal/summer butter, organic meats of all sorts (bison even), and the funny thing was if you went into that same grocery store 2 towns away (new england here) they wouldn't have any of the same organic/natural items. the other 20% of our shopping was the large portion of my budget that went to raw milk, fresh local eggs, farm items, and bulk brown rice for all our baking. that has been done through the UNFI truck, more recently, bulk at whole foods as we moved.
you can place bulk orders through many stores and even ask them to carry items of higher quality from brands that they have. do they have any organic valley dairy? they make a pastured seasonal grass-fed butter!
getting you veggies and fruits and organic meats from the store is a great way to start providing traditional foods on a budget. and even if it's wal-mart, they're definitely shifting their product base to attract the natural foods crowd, and your local store will try to cater to keeping your business. and the more people ask for the natural foods, the more they'll supply.