Well, when I left, DD was 14-15 months old so there was no discussing things with her. Since her birth, XH had only been alone with her for 20 minutes tops, and had never gotten up at night for her. He was of the opinion that she should just be put alone in a separate room and left to CIO, despite the fact that she'd been exclusively breastfed and had coslept since birth. Needless to say, I was freaked out about leaving her alone with him for any length of time, not to mention overnights, which he pushed for from the start, stating that it was his "right".
And FTR, my XH was/is a functional alcoholic, a sometimes cocaine user, verbally and emotionally abusive, with some serious anger problems (breaking things, throwing things, yelling/swearing, denigrating, cutting himself, punching brick walls, etc). He has 4 biological children with 4 different women, and he pays ZERO child support to any of them.
At first, I fought very hard and very stubbornly for supervised visitation, insisting that he had no experience being left alone with DD. Since we were still in the initial stages of separation, and since he hadn't been abusive towards DD, there was no legal justification for supervised visitation...so I supervised myself, despite how uncomfortable that made me. This was without a doubt the toughest part because I had to see XH who kept trying to convince me to give into what he wanted. This is when I went to Al-Anon the most, learning how to enforce my boundaries and to stop being afraid of him.
Generally, I went over to his place after work, once a week, picking DD up from daycare and having dinner with XH and DSS. Usually, by 6:30 pm., XH was already angling for me to leave, probably so he could drink. A few times, he would pick DD up from daycare, but that rapidly died out considering he was super lazy and didn't like lugging her around in the subway (he has no car). On week-ends, I'd come over to his place with DD and spend a few hours there. Gradually (and because I had no choice legally), I'd leave her alone with him for 20 minutes, then 30 minutes, then an hour, always being within cell phone range. In the beginning, I'd just walk around the neighborhood. Of course, XH was trying very hard to prove himself a worthy dad, so he never called me. Eventually, we worked our way up to 3.5 hrs, which turned out to be XH's max. He simply couldn't deal with DD for longer than this.
During this period, I quietly continued documenting the numerous missed visits, the ones cancelled at the last friggin minute or the ones shortened because XH "had things to do". There were also some clear instances of neglect; a few times, DD came back to me soaked in pee, unchanged for the entire visit, dirty, hungry and once she was covered in dried blood after "an accident on the playground". Also, from the start, I facilitated all visitation, driving DD there and back, providing a bag with food, diapers, and wipes, because I didn't trust XH to do what was necessary for DD.
It took patience and lots of vigilence, but I slowly built my case. When it came time to go to court for custody, I had tons of proof that XH wasn't interested in shared custody, nevermind overnights of any kinds (despite stating repeatedly that he had raised DSS on his own which proved that he was good father...I won't talk about how messed up poor DSS is because of his dad). Since XH was afraid or too lazy to go to court, I obtained sole custody by default; XH was granted that same 3.5 hrs a week he always got. Obviously, XH's fear of the authorities played in my favour in my case.
In the meantime, I relied on the support of family, friends, a counsellor and Al-Anon to get through leaving DD with XH. I kept communication to the very bare necessities and through email as much as possible. Whenever XH became unpleasant at drop-offs, I'd bring my dad with me and then suddenly he couldn't act like a UAV. It worked rather well. As for DD, I did my best to always encourage her enthusiasm to go see her father, and always remained cordial in front of her. When her behaviour was off after visitation, I'd shower her with attention.
Eventually, XH became confrontational with me, and since I had been granted discretion over visitation, I told him he couldn't see DD without letting me a) see where he was living and what conditions his apartment was in, and b) introducing me to whomever his roomates were at the time. This caused a huge tiff between us and it resulted in XH not seeing DD anymore. It's been 4 months since he's seen her.
Courts/judges don't care about a Dad emotionally abusing and destroying his kids' self esteem apparantly. 
There needs to be physical violence to make any sort of case against him having visitation... or rather then he'd likely still get it, but I may have more of a case for supervised.
