It is indeed likely to be cold then, but there's still a lot of fun things to do.
I think the Lincoln Park Zoo is fun in any season, and has the added advantage of being free - if it's too cold you can spend more time inside in some of the exhibits (our toddler particularly likes the Regenstein Africa House).
I also think the Peggy Notebaert Nature Museum (quite near the zoo) is a lot of fun for kids that age - very reasonably priced, not too overwhelming, and they'll love the butterfly room and the play area downstairs, among other things. I think they're probably a little young for the Art Institute or Museum of Science and Industry, but I've heard that the lower floor of the Field Museum is great for younger kids (we haven't gone yet, but are planning to take our now-16-month-old there this winter). The Aquarium is right next to the Field Museum, and they might love that. It can be very expensive, though if you don't want to spend $20 (or whatever it is) you can ask for tickets to *just* the Aquarium (not the special exhibits or oceanarium) for $5.
Finally, I think that Macy's on State Street is keeping up the tradition that Marshall Fields always had of wonderful CHristmas windows to go look at (though I'm not certain about that). I have great memories of coming downtown to look at the Marshall Fields Christmas windows in December when I was a kid...
Have fun!
Follow Mothering