doable. not pleasant, maybe, but doable. my mom regularily made a cross continental trip with 3 of us similarily spaced in age. this included two flights of about 8 hours, plus a layover. call your airline and see if they can offer some help with luggage, etc. you might be able to get yourself a nice golf cart ride across the airport too. if your toddler is a runner, you might also consider one of those baby leashes!
anyway, you can do it!