I also believe that breastfeeding has in fact changed my breasts. Sure, pregnancy started it, but breastfeeding has taken a toll.
I apparently am a human cow, my breasts balloon up 2-3 cup sizes when I'm engorged...they never-ever-got that big during pregnancy. I have gotten MORE stretch marks on my breasts since my pregnancy ended (yet interestingly, almost none of my abdomen...) Plus, my poor nipples...maybe this is because I'm a working mom and I pump so much...but my nipples are HUGE...long...discolored... That certainly didn't happen from the pregnancy. It's from 3 years of continuous nursing/pumping.
As far as being "saggy"...I don't know! I guess I'll find out when I wean...but whether that can be attributed to pregnancy or not, I'll never know since I chose to breastfeed.
So I've always found the argument "it's pregnancy, not breastfeeding, that effects your breasts" to be shallow. First of all, it isn't always true, and secondly, if that's the woman's reason for not breastfeeding, she has other issues and is looking for an excuse.