Depending on where you live, there is also a co-op preschool in Sparta. Those will probably be the cheapest, tuition-wise.
If you live on the East side, I think the Franciscan Life Process Center Preschool looks wonderful.http://www.lifeprocesscenter.org/preschool.htm
I've heard wonderful things about it. We took ds2 to music therapy there for awhile. When we thought our Montessori prek was going to be canceled last year, I was considering taking ds2 there, despite the long drive for us. It is in Lowell, but it's the GR side of Lowell, so not as far as you'd think. Again, it's not tuition free, but I believe they do have scholarships you can apply for.
For your 4-year-old, I would recommend filling out a form at your local public school to see if s/he qualifies for the state readiness grant, which will provide preschool at reduced cost or free. You qualify based on income, but there are other factors that make a difference, like if the child was born premature or even living in a rural area. I don't think you're going to find free preschool for a 3 year old unless they qualify for special ed services for some reason (my ds2 qualified for speech, but we ended up just doing 1-1 speech therapy with our local district and then sent him to a different preK).