Its not getting it out of your system that is the problem, its your system getting used to not having it.
Its different for everyone, but 6 weeks is fairly typical. Often the reduction schedules prescribed by doctors are just way too fast and uncomfortable.
Also, its the last part that is hardest (I mean, its fairly easy to go from 20mg to 10mg, but going from 10mg, or 5mg to zero is much harder. There's like a "critical dose" below which the body reacts much more strongly and uncomfortably and usually this dose is pretty low).
It might work better to taper more slowly, and to go to 5mg daily rather than 10mg every other day. Then try to go to 2.5 mg, but if withdrawal symptoms worsen, go back up slightly until they cease, stay at that dose a week or so, then try the decrease again. Some people also use a shorter-halflife benzo like valium or ativan to get through the withdrawal effects - but since you are wanting to TTC maybe you'd want to avoid that.