I just wanted to update this.
After many tears and much agonizing, I supplemented, sobbing as I gave him that first taste of formula.
At least from what we can see now, it was completely the right choice. He was NOT doing well. My mama radar WAS accurate. Within two days I saw a real change in him--much more interactive, alert, and strong. It is so incredibly obvious that I feel bad about not doing it earlier.

He is still getting about 4 oz of formula a day...not very much, and now that I have a new electric pump I hope to be able to increase supply so I can wean him off it. His gain has really improved so much, too--he's gaining much faster now.
I still hate the possible long-term consequences, but baby needed calories.
I want to have this in "the record," so to speak, too, because he was sort of a stealthy not-thriving baby. He didn't cry very much, slept a lot, and wet and pooped okay. (This is because the problem is partly due to excess calorie use at the breast.)
I do have an appointment with a craniosacral therapist. Husband is not really comfy with donated milk, and I do see the concern...while I trust mamas, there is no absolute guarantee of faithful husbands.

Maybe I should get donor milk and home-pasteurize if I can't get past this.