My experience with yeast was like an icepick deep into my breast, a sharp shooting pain, the outside of my breast didn't hurt at all (except my nipples, which felt like they'd been through a cheese grater).
I think yours sounds like Mastitis - sudden onset, concurrent fever, outside of the breast. I would either look into homeopathics (I don't know the name, but I know a couple of people who think it cured their mastitis), or take the antibiotics with acidophilus etc. If it is mastitis, you will have to treat it first, and deal with any possible yeast aftermath later. (Mastitis untreated can lead to abcess.)
Best of luck.