For me, dairy and eggs (even though they were almost exclusively pastured) didn't do it for me. Fish is nice, but doesn't cut it for me. The only thing that really made a big difference for me was red meat: specifically lamb and beef.
As for the expense, IMO, the most cost-effective way to get good beef, etc. is to buy directly from the farm. If you are just wanting to try it without a long-term commitment, then buy a few pounds from the natural foods store. If you really want to eat it long-term, find a farmer and buy it from him. Ask if you can just buy a few pounds, or by the quarter. If you have to buy a larger quantity, find someone to split it with. IMO, even if it's not certified organic, if it's pastured, it will be far and away better than anything you buy in the store. Many farmers can't afford organic certification, but they follow sustainable farming practices, and are very close to organic anyway.
A point of reference, my family of 7 gets a 1/2 beef every fall. It lasts us more than a year, as we eat as many plant foods as we do meats. It's all in how you prepare it!