I spent 2 years working evening and weekend shifts. It put a lot of stress on my marriage and I found I was getting fewer and fewer invitations from friends (since I could never hang out with them, they found other people to hang out with and slowly forgot about me. It is natural, really.) Quitting that job was one of the best decisions I've made, even though it was a hit financially.
I guess I am confused too about what you'd be getting or giving up, but I'd say for me, the time with my loved ones was much more important. On the other hand, for a lot of couples money is the big stressor of the relationship, so if that is the case for you, maybe you should pick the money. You could also give it a go in the new position for a year or two and then try to arrange for something better if it doesn't seem to be working out.