Are you sure this is what you want? In my experience, all relationships come to a point where passion subsides, but it can also come back. The initial passion in a relationship is intoxicating and addicting, but it isn't real. Perhaps you should try councelling to help you work through all your confusion. It sounds like this relationship would be worth it to try to look deeper into why you are feeling like having him leave. Another question you may want to consider is how would you feel if your husband eventually moved on to another relationship?