"You Are Your Child's First Teacher"; "Homeschooling Our Children, Unschooling Ourselves"; Home Education Magazine and website; author John Holt; "Creative Play With Your Toddler"
Inspiring books unrelated to homeschooling, yet somehow fit in: "The Tracker" by Tom brown Jr; "Farmer Boy"; hmmmm, I'll think about that too. There's more, I just have to think.
Places and groups: Library (check the events calender); Zoos (same, check for special events); Junior Ranger program offered by the Nat'l Parks Service; 4H; special focus community groups; festivals (bluegrass, mushroom, salmon, Cambodian, whatever); Mountaineers; etc etc etc.
This is off the top of my head. Our family is unschooling. When they were little I was inspired by the Waldorf philosophy, but I was too unschoolish to dedicate myself to it. (There are home-based waldorf curriculums available online. Oak Meadow and Christopherus are 2.)
To save money, avoid curriculums unless you have very good recommendations from someone you know. And find out about the laws in your state. We don't have to "declare" until 8yo, and oversight is minimal. This allows us the freedom to educate as we see fit (and save a whole lot of money if we need to.) I am a big fan of "delayed academics": a fancy way of saying that it really is better to let kids be kids!