A-ha! I know exactly what's wrong.

What kind of detergent are you using? Nearly all the mainstream ones (even the "free & clear" ones) contain optical brightening dyes now. These are designed to coat the fiber strands, and leave a purposeful residue that reacts to light and causes colors to appear brighter.
The problem is the residue provides an ideal breeding ground for bacteria, and the more absorbent something is, the more of the dyes it holds onto (so cloth diapers and towels are big culprits here).
And while a towel may smell clean out of the washer/dryer, once it's exposed to moisture, that released the bacteria smell, which for towels is musty. (For diapers it's ammonia -- it's because of the urine that activates the bacteria.)
Using more detergent will make this WORSE. Using vinegar will help in an ongoing fashion, but won't remove any residue already there. Oxiclean might help break up some of the residue but more than likely in a HE washer will bind to the existing residue and then you'll have more issues.
Get some blue Dawn liquid dish detergent -- it has an ingredient that breaks up the residue. I would add about a teaspoon to the wash water (dissolve it and add to the bleach dispenser, or squirt directly into the tub before adding towels). Add towels and wash on VERY hot. Rinse repeatedly until water is clear and no bubbles.
Then switch detergents! And never use a fabric softener with towels. That makes them less absorbent and more likely to get bacterial buildup.
For a front loader HE washer you're to use about half the recommended amount. For most mainstream detergents, that's actually about 1/4th the recommended amount (they recommend too much typically, because then you use more and buy more).
Some brands that are truly water soluble (meaning NO residue is left behind):
Planet
Charlie's Soap
Ecos Free & Clear
Arm & Hammer Essentials
Country Save
Maggie's Soap Nuts
Mountain Green
Nellie's
Seventh Gen Free & Clear (contains enzymes -- need to use with hot wash following to prevent skin issues)
Sun & Earth
Trade Joe's
If you can't find any of those online or in a store near you, PM me for more suggestions.
If you're already using one of the water soluble ones, and the problem started after you moved into your new place, then the washer itself has buildup. If that is the case, go get some Simple Green (auto dept, Target) and an old ratty towel. Add about a tablespoon of SG to the washer (again diluted into bleach dispenser or direct into tub) and toss in ratty towel. Run hot wash and one extra rinse to clean out the washer.
Good luck!