There's a nice park in Montpelier, Kellogg-Hubbard Park, with walking (and I think biking, but I'm not sure) trails that go on for quite a ways. There's also the North Branch Nature Center near town, which is great - it's right by the river, there are fields and woods and wetlands.
For something more touristy, there are a lot of things in Waterbury along route 100 - Cold Hollow Cider Mill, Ben and Jerry's factory, and so on. Many are free or low cost and have food samples.
There's also a nice history museum in Montpelier, next to the state house (admission costs though). The State House itself is beautiful and worth a visit, and it has a large lawn that could be a good picnic site.
There's a local newspaper, "The World", that has a good list of local events.