Thrift store sheets or tablecloths are a great source of cheap material.
If you're just starting out though, I'd recommend buying the cheapest muslin you can find for practice. Be aware that muslin comes in widths from 24 inches to 120 inches, so take that into account. But it's great to practice stitching straight, following a curved line, etc. Use a dark thread on it to see what you're doing better.
Definitely check out the remnant bins at Joanns. I always find at least one thing to buy, and they're usually at least 50% off regular price. They're relatively small pieces, but for just starting out, that's okay. My first rule for fabric is 100% cotton, so pretty much if it's cotton and in the remnant bin, I buy it. I was bummed when our local Joanns closed, but my husband was thrilled (he's been trying to find storage solutions for my stash).
We also have a discount fabric store here. Kind of like a Grocery Outlet for fabric. You can never count on finding something in particular, but there's always something interesting and cheap. Look around and see if you have anything like that around you. Also keep an eye on Freecycle and Craigslist for people de-stashing - just be sure it's a non-smoking house or the fabric will reek.