Hi Crystal- Honestly, I would feel the same way. What I would do is just come out and ask him if he would call you when he is leaving early. Unfortuantely, men can seem/ be clueless when it comes to how women feel about these things, it's usually lack of relationship experience, but some men never get a clue. He will likely never figure it out on his own and it's best just to be direct. Do try to leave out any wording that might make him sound inconsiderate.