I keep old utility bills for one month. I don't keep credit card statements at all--I shred them immediately. Both my credit cards have all that info online. I do keep bank statements, but I'm thinking I don't need to do that, since I have been able to retrieve old info that I need faster online.
If you're going to toss the stuff, make sure you shred it well!