I'm also on a tight budget but with a little effort, I've been able to get us switched over to about 90% organic food, etc. Here's the summary of what I've done:
Food: The key is menu planning, cooking from scratch and buying in bulk. It sounds like a lot of work but if you organize a bit (and I'm not an organized person) it's really not hard. It's also a whole lot healthier than prepared foods. Stay out of the organic food section of the grocery store and don't buy pre-packaged and prepared foods! To give you an example, I can buy one pound of dry organic beans for just over one dollar. A pound of beans makes 6 cups of beans. One can of organic beans is over a dollar and I think it's a little less than 2 cups of beans. So right there, you can reduce your costs to one third. Beans just have to be soaked overnight then cooked for a few hours in a crockpot or on the stove. Make a bunch and freeze leftovers for easy future meals. Oatmeal, rice and other organic grains are also affordable once you buy in bulk.
Try to find a co-op, buying club, etc. This makes a huge difference. I don't know where you live but I go through Azure Standard (
www.azurestandard.com) and they deliver to a large portion of the country.
Check out cookbooks that have 'from scratch' recipes for ideas. Nourishing Traditions is one and I've heard good things about Feeding the Whole Family.
Meat and eggs: Buy from a local farmer. Check out
www.eatwild.com. Buying 1/4 cow (the butcher cuts it up and packages it for you) and freezing it will save a ton of money. Plus pastured beef and chicken is so much better for you. Also, if you have a local and independently owned health store in the area, they might know of local farmers. Also, just eating less meat (something we're working on) lowers a grocery budget. Beans and eggs are cheaper than meat. Remember you can also make your own stock with veggie scraps (just put leftover peels, stems, etc. in a bag in the freezer until you are ready to make stock) and leftover chicken carcases or beef bones. After I pick a roasted chicken clean of useable meat, I dump it (the skin, bones, everything), the bag of veggie scraps and 2 T. of white vinegar in a pot. I let it sit for an hour then bring it to a boil. Next turn it down to simmer and let it sit for 6-24 hours (longer is better). Ta da! You have the tastiest, thickest, and healthiest organic chicken stock around and it's essentially free since you used scraps. Saves $1.75-$3 a quart for organic chicken broth. I usually get 6-7 quarts of stock from each batch.
Milk: Milk products are tough, but because they are higher up on the food chain (as the PP mentioned), these are worth the splurge IMO. I actually buy raw milk from a local farm and make my own butter and sour cream. This is not nearly as hard as it sounds

If you are interested in raw milk, check out
www.realmilk.com.
Veggies & fruit: Try farmer's markets or Community Supported Agriculture (CSA's). With a CSA you buy a "share" of a farm and get a box of the resulting produce each week during the growing season. CSA's can be found at
www.localharvest.com. Also, keep an eye out for farms that are using organic practices but aren't actually "certified" organic. The certification process is very expensive so you can save money by getting to know a farmer and their practices. Also look for U-pick farms this summer and freeze berries and veggies. Canning applesauce and pear sauce this fall when organic fruit is in season also saves money and gets you set up for fruit all winter.
Cleaning Products: I have switched to cleaning only with white vinegar and baking soda. That's it. Extremely cheap and effective. I keep one spray bottle with straight vinegar for the bathroom and one bottle with half vinegar and half water for everything else. Baking soda is a great abrasive so it can be used for bathroom tile or the toilet. For the toilet bowl, pour in some baking soda, swish around, pour in vinegar (it will fizz a bit) and then brush vigorously. So far I haven't found anything that baking soda and vinegar doesn't work for. An added bonus is that I can clean around my baby!
For laundry soap, dish soap and dishwasher powder, I use BioKleen since Azure Standard has a great price on them. I buy a 10lb. box of laundry soap and it lasts around 100 loads (for about $11). The dish soap is lasting forever and I only use 2 teaspoons of dishwasher powder instead of the recommended 1 tablespoon. My dishes come out nice and clean but I also have soft water so if you have hard water, I think you need to use more. I get about 80 loads from one can of dishwasher powder (about $6).
Use cotton cloths to clean instead of paper towels or those disenfecting wipes. Also, cut down on paper towel use and you might be able to afford recycled paper towel and toilet paper products (such as 7th Generation brand). Check out your local co-op or Amazon.com for good prices.
Body Products: Check out this forum. There are lots of good ideas including going "no poo" which basically means not using shampoo any longer. Instead, wash hair with baking soda and use apple cider vinegar as a rinse. It works beautifully! I use coconut oil (the same stuff I cook with and buy by the gallon) to style my hair and prevent frizz. For soap, check out Doctor Bronner's bar soap for the bath and liquid soap mixed with water for pump soap.
It's a lot to think about but try implementing one thing at a time. It's completely worth the effort!
P.S. If you happen to live around the Portland (Oregon) metro area, check out
www.sustainablebudget.com. It's a fantastic workshop series on eating organic locally produced food on a small budget.