There are so many variables here. Do you/can you garden? On a big scale/tiny scale?
Can buy from farmers, produce stands, CSAs, co-ops, Upick? Sometimes these places will let you work in exchange for a discount or free seconds. Yes, you will get more produce than you can eat before it goes bad, but whether you are working or just buying, to be cost effective you need to freeze/can.
Do you have any means of free (or nearly free) food- nuts, berries, apples, swaps, bartering, hunting, etc?
Do you have any food animals, or the room for any?
Personally, I don't think you can achieve a low food bill exclusively shopping at grocery stores. You have to go to restaurants, or bulk suppliers- not like Costco or Sam's, but more like Aramark or Sysco- or, best yet, straight to the grower.
We have no food animals, but we do everything else I mentioned or we cooperate with someone who does them.
I love Costco. I have never been to Trader Joe's. I don't AT ALL mean this as a criticism of either of these stores or of your shopping methods. I just think that stores *have* to mark up to stay in business, and typically, big warehouse type stores don't even have any loss leaders.