We women are so considerate. We owe dads nothing, and yet we sacrifice our aspirations so they can pander around with the kids every other weekend.
Weigh your sacrifices carefully. Of course your kids will want to see their dad. Maybe whether they do or not is up to him. You are their primary caregiver, and so if the inclination took him, he'd be free to move anytime he wanted and anywhere and no judge could stop him. Ask yourself why you are moving. It doesn't sound like it is to be spiteful...it sounds to me like you might be going back to someplace you left because at the time you were making sacrifices for a relationship with him. Talk to him about it, tho.
Being sneaky about it is no good. Of course we all strive to co-parent healthily, and that would be great if we got to co-parent with strangers instead of our exes. You asked about what happened to those of us who left; well, I'm so glad I did, and for me and my baby its been the best thing. I didn't leave to punish him, but to give myself the possibility of solace and happiness. Distance has been good for us. But as she grows, if he stays where he is instead of moving to us, I know she'll idealize daddy, feel sorry for him, think I'm mean, never ever fight with him (you have to be close to do that) and want to move in with him one day. So weigh it all carefully.