The rural area I live in has large Amish and Mennonite communities nearby (in different directions). We get many of our staples from Yoder's Bulk Foods (Mennonite) and many baked goods from Yoder's Bakery & Greenhouse (Amish). There are places in these communities where one could buy eggs, chicken, milk (raw), strawberries, in season, and bulk foods like flour, sugar, honey, sorghum, spices (great prices on spices!), boxed cereals (which I don't buy, but have seen), nuts for cooking, dried beans, lye for soapmaking, really a large variety of things. One of my daughter's favorite cups came from an Amish bulk foods store. It's stainless steel and about 6 ounces.
The other thing we do I we found a local farmer on localharvest.org
They are 2 1/2 hours away, but every other week they drive to a city that's just 1 1/2 away and we can pick up groceries. No, we don't go every other week, but we could if we needed something. They provide us with beef, pork, chicken, turkey, some supplements, and some staples.
We do still buy some things from Kroger (orange juice, coffee, etc.), but rarely go to Walmart (paper towels & toilet paper) anymore. Before this, those were my two local (in the next town) grocery stores.
We also found a quite local egg lady. She lives just outside of city limits and has wonderful green, brown, and pink-ish eggs for $1.50/ dozen! And she's a wonderfully sweet lady, to boot!
Yes, it takes more time to shop this way, but it is real. My kids are learning that some days there simply aren't any eggs. That they don't just come from some magical place in a fluorescent lit store. They have met the chickens. It's a good experience for us all.
If there were a WholeFoods or something similar closer than three hours, I'd probably make a monthly trip there, but I'm happy with our solution so far.
Oh, I almost forgot to add farmer's markets in season! Great resource!
And I have just started researching Alice.com
I'm not through comparing prices yet, but it looks interesting. Free shipping on everything. Even if it's a tad higher, it might be worth it for my "Walmart stuff" since taking to kids to Walmart is quite an ordeal!