When I was growing up in New Zealand my parents owned an Angora rabbit farm. We kept the rabbits inside, in elevated cages, to keep their fibre clean, and my mother would share the fibre from the rabbits and grade the wool, which was pretty time consuming, but if you only had a few rabbits you wouldn't have that issue (we had 150 of them!).
If you do go ahead and get a few, make sure you get all doe's because the bucks tend to 'spray' a bit when they pee, and sometimes it goes on there fibre and it is then unusable - (also you don't want to start breeding your own by having a mix of bucks and does!)
We feed ours a diet of hay, pellets, dock leaves and carrots. They do tend to get pretty smelly, so make sure that what ever kind of 'shed' you build for them to live in is well ventilated. Ours where in cages elevated about a metre off the ground, and we would 'muck' out under there cages every month or so (there manure is fabulous for fertilizing your vegetable garden, esp for growing corn!)
Angora rabbits are very timid animals, and not suitable to keep if you have dogs (as they can easily go into shock and die from just the sound of a dog barking) they love to have music constantly playing in the back ground, it helps to keep them calm (we always had a radio with classic hits on playing in their 'shed'

)
Feel free to PM me if you have any more questions
poppyseed