Quilting can be really cheap. I made dh a quilt for xmas out of recycled cotton check shirts. I went to our local op shop & got the equivalent of $20US worth of checked shirts in funky colours.
You can overdye fabrics to give them a new lease of life & change the colours too which can be very cheap to do if you use something like a procion dye where you can only mix as much as you need to use. Tie dyeing recycled clothing is a cheap craft. You can also cut up your tie dyed clothing to make patchwork. With overdyeing & tie dyeing, you can get real interesting effects depending on what your base fabric was. Once it gets cut up into patches, it then can get even more interesting.
Often you can find really grotesque clothes in op shops for next to nothing that are made out of exquisite fabrics that work out way cheaper than buying it new by the yard. You can get lace & ribbons for projects off old clothes too.
I've found getting my hands on pure wool sweaters to unravel & felt second hand quite difficult. But I do have a local sheep farm near me that sells uncarded fleeces & natural coloured wool at very reasonable prices. So I buy these & dye them. You can then felt the fleece, or spin it & knit / crochet things. I just made a massive 200g ball of rainbow coloured wool using 200g of natural cream wool ( $4.25US ) & a mixture of acid dyes.
Natural dyeing is another cheap option. Well I think it could be. I've not got 100% into it yet but it looks cheap as all the materials are free. Just not so sure how colour fast it all is.... but I s'pose if you were making something like wall hangings that didn't need to get washed often or get exposed to too much UV light you could be OK. Altho', that said, a friend of mine actually has a very cool hand spun, hand knitted & hand dyed from local native plants sweater that is really beautiful. Very muted & subtle natural colours.
I've got this awesome book called something like Young Crafts which I got years ago in a book sale at my local library which has all sorts of cheap craft ideas in. Stuff like making copper bowls out of recycled sheets of copper. Making drop spindles. Making simple looms & how to use them etc. So check out your local library for more ideas.
I got a really good book out of the library the other day on making containers & baskets out of recycled materials. But I guess just a regular book on the techniques could be adapted to whatever materials you can find to hand.
Flax weaving. Not sure if New Zealand flax ( Phormium Tenax ) grows where you are but you can do some awesome weaving & basket making with it. You've probably got lots of natural materials you can make baskets & mats out of growing locally. Again your local library probably has books.
On the subject of rug hooking, I saw an article in one of our mags here about this woman in England somewhere ( I think ) who makes textiles out of recycled things. It had a sample project which was a purse out of strips of plastic carrier bags. Collect interesting colour plastic bags, cut them into thin strips & then hook them thru some canvas fabric like you would make a wool rug. There are a few crochet patterns on the net for bags & hats & sandals made from recycled carrier bags you can find with google.
Collect objects off the beach, if you live near one, like shells & sea glass & drift wood & interesting bits of sea weed & pebbles. We've got a beach at the bottom of our road that used to be a very popular picnic spot in the 1910s. It has bulk bits of old broken china on it that you can find if you look hard enuf. You can use the things you find to make all sorts like jewellery, mosaics, mobiles, wall hangings, ornaments, pavers for the garden etc.
I think hypertufa could be a cheap hobby too for garden sculptures & pots. I've not done any myself due to there only being 24 hrs in one day but have seen recipes for it & it looks quite interesting & cheap. Well it's gone on my list of to be investigated things later on