At some point I decided that for my own mental health I was going to have to let some things go. I had to decide what I was comfortable with, share that with my husband, and then let things play out however they played out. I removed myself from the process. So in your situation, I might have told my husband "I don't mind taking them to swim lessons, but I can't do it on the 11th. If [ex] can take them that morning she could drop them off at the library with me afterward, or you can pick them up at 7:45 as usual and they will just miss that week. Let me know what you guys decide." Then, because my husband struggles to remember details (or because I am overly controlling and it is part of my coping skills, whichever...) I will email him an email to send to his ex, written in his voice, which he can read, forward, re-write, ignore... whatever. It gets it out of my brain and gives him the information he needs from me.
Then I would just leave him to deal with it. He might forward the email, write a different email, argue with her about it, make a decision about which option he would take, opt to give in to her $8 demand, take time off himself... But I've given him my bottom line, which is that I can't take them to swim lessons on the 11th. Whatever he comes up with, as long as he respects my bottom line, I have to just be okay with. I have to just trust him to make a reasonable decision, even if it is different than what I would have come up with.
It took me a lot of practice and a lot of self-talk to let things go. Sometimes I have to imagine made-up scenarios about being someone's nanny or a kindly neighbor transporting someone else's child or something, anything, to trick my brain into letting it go. And it certainly didn't happen overnight. But I didn't like the person I was becoming, and it was negatively impacting my relationship with my husband, and I was allowing his ex WAY too much space in my brain. I can't control her, and I can't control their relationship, and I can't micromanage their interactions, but I can control my own behavior and my own reactions, so that is what I focused on. At the beginning it probably took at least as much energy to let go as it did to be involved, but now it has become our habit and I am MUCH less stressed by the whole thing. Yes, sometimes he makes decisions I think aren't the best ones, sometimes it feels like I am being inconvenienced unnecessarily... but so long as he respects where I draw the line for myself and the kids WE have together, I just have to accept that those decision are his to make and that he is making the best decision he can with all the information he has. If I am extra annoyed by his decision, I usually step back and take a "big picture view" and ask myself how much this 2 hours or $20 or one concession is going to matter 5 years from now... and that is usually enough to help me move past my annoyance.
Good luck. I know "let it go" is much easier said than done, but I can't adequately express what a HUGE difference it has made in my life.