I think the real cost savings is whether you own a car or don't own a car. Once you are paying for the expense of ownership (insurance, registration, and possibly payments) and maintenance, it's often most cost effective just to use it! However, if you go car free, depending on your circumstances of course, that's when you can start to see savings.
I was car-free for 5 years. We were given a car last May and are now waffling between, wow, isn't it nice that we can just go wherever we want and whaddya mean it's yet another $500 repair?! It broke down last week, and my dh and I were both like, geez, we hate cars. We're still figuring that one out.
The overview: Obviously a big help is a functioning mass transit system. For most large cities this includes a subway. I've ridden a subway a few times, but have basically zero knowledge on how it works, especially as far as cost. In a medium-sized city, the bus is your best choice for getting where you need to go in a hurry, especially with children. The city I live in now has an unlimited rides pass for $60. This makes riding the bus very affordable. Kids under 5 usually ride for free, or for a reduced fare.
Taxis can be helpful, though they are pretty expensive. Getting between cities, greyhound/megabus and amtrak are often affordable. Renting a car on occasion can be useful for long-distance traveling.
When I had no car, I took the bus to the grocery store, bought roughly a month's worth of staple goods and a week's worth of perishables. I took a taxi home. I lived three blocks from a (quite expensive) corner grocery, where I could buy decent produce and discount bread. A thing of flour was like $4, so it was worth it to go for a big spree once a month. Also, in decent weather, I could bike to the regular grocery store and haul home food in my bike bags and in a back pack. I think it was about 1.5 miles away--very doable. I've also heard of people using a bike trailer to haul home goods. I think that'd be easier, because a bike gets hard to balance and pedal with 30-40 pounds on it.
The bus in the two cities in which I've lived goes downtown, and I try to make sure my bank is downtown, and the library too. That way there's no transfer, just one trip downtown. Sometimes things are more expensive downtown, but if you're there, it sure beats an hour bus ride to the edge of town to a big box.
A lot of times, we just had to skip things happening at night, as our city had no night bus for a long time, then established an inefficient system--like it would take us 2 hours to get home by bus, when it took about 7 minutes by car, or 45 minutes to walk. That was a bummer, as was not being able to visit my folks in my hometown. They only lived an hour from us (by car), but it was rural and there was no mass transit.
Biking is pretty cool, though I imagine with little kiddos like you have, it would be more of a challenge! I'm going to soon look into getting a bike trailer, because I have a 4 month old. When my daughter was 4 or 5, we got into biking, and we were blessed with a tag-along, a half bike that attaches behind an adult bike. The kid can help pedal, but can quit when she's had enough. I think it's helpful until they are ready to bike on their own. At age 9 or so, she was biking on her own bike with us, going a mile or so at a time.
I've heard of hard-core bikers pedaling through all kinds of weather, even snow and ice (my current dh is one), but I have cold-induced asthma, and it's made worse by car exhaust, so for me, once the weather gets below 50 or so, I'm off the bike.
And of course, there is walking, which I really enjoy. I can walk a mile in about 20 minutes, which I think is a pretty reasonable distance. To put your own house in perspective, you can mapquest what is a mile away from you. Then on a physical city map, make a circle that distance around your house with a compass. You now know what is a 20 minute walk, or 5 minute bike ride away.
My daughter *hated* walking as a kid, still does. She would run like crazy with her friends, but ask her to walk anywhere, and it'd be a constant whine. So we started doing "surprise walks", where we are walking, but totally distracted by paying utter attention to our surroundings, looking for surprises that were left for us to see. For example, we'd see a smiley face spray painted on the sidewalk, a spinner hanging in a tree, and so on. Obviously it was stuff that was already there, but she'd get a huge kick out of it. (She was about 4-5 then.)
So that's far from an exhaustive list. Anything to add? Any questions on specifics?