I ditto the recommendation of knitpicks for inexpensive yet good quality yarn. You can get a ball of good soft wool for $2, or superwash wool for less than $4. They also have cotton, alpaca, and a couple of silk blends. They mostly have wool and wool blends though.
I usually get my fabrics from joanns, or joanns.com They have sales fairly often, and if you get on their mailing list they will send you fliers ever 3 weeks or so, including a 40% off coupon which can be used on any one item (including a single cut of fabric of as many yards as you want). If you have a joanns near you, and can spread out your purchases, I'd just get one thing with each 40% coupon... that's waht I always did when I lived near one

You can also use the coupons online, but you do still have to pay shipping there.
Walmart often has a lot of stuff on the $1/yd clearance table, and some $2/yd too. You have to inspect the fabric though, because some of it is junk.
Depending what you are wanting the fabric for, I sometimes get old sheets or curtians from second-hand stores. At goodwill I could get a queen size sheet for $1, and that has a lot of fabric in it! I use those for linings, dress-up stuff, and working out mock-ups (altering patterns and stuff). Old curtians are nice heavy fabric and are great for dress-up clothes, skirts, throw pillows, or

new curtains.

I also know people who find sweaters at thrift stores and then unravel them to use the yarn for new projects...a second hand sweater will often be just a couple $$, and yet if you buy new yarn a sweater is $30-40 of yarn, so that's pretty cheap.

My last recommendation would be to check out
www.etsy.com and search under supplies. There are people there who do the unraveled sweater yarn, or who sell off from their personal stash at reduced prices. There are also people with hand-dyed yarn and things like that (so they're on the expensive side), but you can set it to show results by price, and there are always a few really cheap ones in there.
