This wouldn't fly in our house.
I'd be willing to sit and talk together and work through it, find some better alternatives - on the condition that he apologized to our child for what he did. I would work towards a solution - but I would only do that once. If the same sort of thing happened again, he'd be gone.
And where I live, soap in the mouth is abusive. I wouldn't hesitate to contest his rights to visitation until he's proven that he's making steps to improving himself.