He may or may not get better, but you can minimize how it affects your life. When he refuses to bring the kids to camp, what happens? That is the kind of thing where I would completely disengage. My daughter was in Irish dancing so I told my ex the time and place, that she'd love it if he made time for her to attend during his visit with her, she'd love to show him what she can do. Then I didn't mention it again, or make any effort to work around the class time. If he had the kids on that day, he could take her to class or not, it was up to him. He did take her there once, but dropped the ball after that. She wasn't thrilled about missing so many classes, but that was between him & her.
How old are the kids? Mine are on a trip right now and we arranged for them to call me on certain days, in the morning. They knew that after 2 sleeps it was tiem to talk to me, so if he 'forgot' they would remind him. I told them it's okay if they don't feel like chatting a long time but they had to at least say "Hi!" because I'd be waiting for their call. My kids are 10 & 12 and I don't know whether my ex helped them initiate the calls or if they reminded him, but they have called.
Drop off times is tricky... my ex tends to say one time, then bring the kids back early and we won't be home yet. Then he texts my husband all in a huff and tries to make the kids think we forgot about them. They are too old for that, they can read the time so it's pretty ridiculous. We just continue to make plans where we'll be back at the house a half hour before the scheduled return time and if they beat us there, oh well? They can visit in the yard until we get home.
The children and families thing is scary, is that going to deter the other family from having your son over? That, I'd probably keep records of if you can. I don't know what can be done about it, but if he's trying that hard to control you that he's willing to ruin the kids' social lives then maybe he should have reduced custody, or even just have it written into the order that you don't need to disclose their location to him, as long as they're in town, or something like that.