Oh goodness no, or we'd never leave. One of my kids spikes fevers for *everything*, even allergies. She spikes *high* fevers too--105-106. There was a time where she spiked a fever every week for several months (she had frequent ear infections and would always spike a fever with them). We took her in the time she had febrile seizures, and if it's unusually high for her or doesn't go down in 24 hours (her pattern always has her fever breaking after 24 hours).
The baby has an immune deficiency and gets sick a ton too, so we can't take her in every time she has a fever. We'd have to pay rent there....

:
Basically, I treat at home unless it's a) an unusual fever that isn't normal for them and the illness accompanying it is very acute and/or severe (for example, my son had a 106 fever in October, and it turned out it was from h1n1. We had a very young baby at home at the time plus his school required a note, so we took him in for that fever). B) Accompanied by seizures. C) Doesn't go down right away if medicated or within 24-48 hours if unmedicated. D) The child is dehydrated or getting close to dehydrated.
So basically, a typical fever with a mild illness, I let run it's course. If it's severe or looks like it is getting dangerous to the child, I take him/her in ASAP. Oh, and if there is a fever and the child is complaining of a sore neck or the child looks like they have a stiff neck, I personally would go straight to the ER. That's one of those that I won't wait and see longer than a couple hours.