My SIL is trying an approach of "play and leave" - she sits with her DS (2 yo) and starts playing with him. When he gets really involved in the task, she quietly walks away and starts another task (loading the dishwasher - something she needs to get done). When he comes and gets her or fusses for her, she goes back and does the same thing. He has learned that she will be there - that if he asks for her she'll come - so he's less anxious about her ditching him, and he's learning to play on his own more. She also tries to encourage her 4yo to play with him also, and that is helping. Over the past few weeks, he's been willing to play alone for longer periods of time.
Good luck - sounds tough. But this too shall pass. It's great that he's okay with other people - the sitter/his father.