Here's something from OSDS:
Preparing Wool Knits (such as jersey and interlock) to make effective diaper covers:
Never use regular detergents on wool.
Wash your wool on HOT with Eucalan® Wool Wash, then dry it on hot in your dryer.
You may want to repeat the process to ensure that the wool is completely 'felted'. Shrinkage varies quite a bit depending on how hot you water and dryer are, and how rigorous your agitator is. You can lose 12" in length and 8" in width so be sure to buy enough. (Wool interlock shrinks quite a bit less than wool jersey).
Your wool is ready to cut and sew into a beautiful wool cover!
After the cover is sewn, it's ready to use. Air it out between uses. You won't need to wash it in Eucalan again for 1 - 2 weeks or unless the diaper leaks poop on it.
This wool can be washed on a gentle cycle in cool water with a capful of Eucalan®. Lay flat or hang to dry.
If I were you, I would felt the sweater before making the pants - remember that wool shrinks when you felt it. Don't know about cashmere, though. I wonder if it will be substantial enough to survive? That's just me wondering, though. hth