Coming late to the party (that seems to happen to me a lot!). I'm so sorry this situation has gotten so complicated for you and your family.
Let me be blunt: you need help. Lots of it. These situations can get so convoluted, so ugly, so heartbreaking, that no family, no matter how intelligent, well-meaning, loving, etc., could ever hope to untangle things and make them work again. You are right that your SD is very manipulative, but she doesn't know or understand what she's doing. She can only do what works for her, to make her world feel as secure as she can possibly make it feel. Sounds like BM is unstable; it's hard to pin an unstable parent down and make them be the rock that a child needs a parent to be. So she's taking drastic measures. I'm glad you've chosen not to give her an ultimatum. My DD's BD gave her an ultimatum once (having to do with something much more minor than this, but she hurt his feelings and he struck out) and it wounded her deeply. Her (and she's 11 too) knee-jerk reaction was to reject him because what he said was so dramatic that it frightened her, and she just wanted to run to me for the sake of her emotional security. It's far, far too big a decision for an 11 year old child to make. Her behavior is motivated by something, and until the something is removed, she's powerless to change it. Your SD needs help to make this change.
OK, now for my little speech about the concept that is so dear to my heart: disengagement. Learning to disengage saved my family. I was so incredibly wrapped up in rage and resentment at SS's BM, as well as anger at DH and SS, that I was barely functional. I thought DH wouldn't even try to parent SS if I didn't push him. He was practically paralyzed by his guilt over the divorce. I believed I was holding everything together. But when I was practically suicidal over the whole mess (and it was messy; w/o boring you with all the ugly details, there were child abuse accusations, legal threats, quasi-stalking behavior, the whole nasty drill), I read about disengaging, got myself some professional help, and took the leap. No exaggeration: I believe that that action is the reason my family is intact today. I am at peace. DH and BM are SS's parents. They make the decisions. They bear the responsibility. I love my SS, but we have a relationship that is not parent/child. Hard to describe, since of course I am an authority/parental figure in his life. If you met me at a party and asked how many children I have, I would answer "four" (a number that includes my SS). But I'm not his parent. The best advice I can give you right now is step out of this situation
. She's not your kid, this is not your battle. You can be your DH's sounding board, support, advisor, whatever, but his co-parent of his DD is his ex-wife.
I think mediation could really help your DH and BM to sort things out, but all you can do is suggest it to your DH. The best thing to do right now is to lay down the reins. If DH doesn't pick them up immediately, breathe, do yoga, walk around the block until you pass out, whatever it takes to let those reins lay. Your DH will probably pick them up eventually, but even if he doesn't, this isn't your battle.
I wish you the best. I feel so heartbroken for you; I remember all too well how much this stuff hurts. We even had some ILs involved in our situation (they got in on the abuse-accusation act), and there's little in life that has hurt me worse. Hang in there, and know that you're not alone.