From the pregnancy book I was reading (was looking at this the other day, since I'd been feeling sick myself):
Both animal experiments and studies in pregnant women have shown statistical increase in spinal column abnormalities in mothers who had prolonged, elevated temperatures above 102 F (39 C) in the first trimester, especially between the third and fifth week of pregnancy [this book counts weeks from time of conception - so that would be between weeks 5 and 7 if you count from LMP]. These studies looked at women whose body temperatures were elevated from extended hot tub exposure; the up-and-down fever pattern associated with most infections is less likely to harm baby. Nevertheless, it is prudent to treat a fever aggressively.
But, from the facts that your fever was at 2 weeks gestations - before neural tube development started - and it wasn't that high of a fever, and it didn't stay high for long, I think you should be fine.