I strongly recommend waiting at least a cycle or two post-removal before TTC.
I had my Mirena removed last September and got a positive hpt two weeks later, so I must have been ovulating right at removal. It turned out to be ectopic, it was very scary and lost my left tube as well. I feel certain it was caused by the residual affect of the Mirena and would not have happened had I waited a few cycles.
Luckily and happily, I am now 24 weeks pregnant and all is well so far