Conventional sexual morality is an attempt at a solution to the question of "how do we make sure mothers have some help in the job of raising children?" It is not the only possible solution, or even necessarily the best, but if you step outside of it you have to find your own solutions. Also, I suspect the poly lifestyle attracts people who don't want a committment. Two lovers are not necessarily always better than one when you are looking at parenting.
Look at it from your girlfriend's perspective. Did she want kids? She is not biologically the child's mother, she is not your legal wife, she has never met the kid and thus has had no chance to get attached to it. She did not make the "mistake" that brought the kid into existence. She still has the option of walking away and continuing her single lifestyle.
The bio father doesn't necessarily have this option. He bears some responsibility. (If you don't want to be a father, wear a condom). The child is biologically his. On the other hand, if he takes the job of father seriously he is in for a lot of work. Being a parent can cramp your dating style. I get the impression that you are less serious about the relationship with him than you are about that with your girlfriend. He is looking at the prospect of being permananently tied to a lesbian who isn't really interested in men. (lots of men fantasize about lesbians...but the reality of dating someone who isn't that interested in men has draw backs in reality)
For that matter, I understand some lesbian couples don't want the men involved, they want to raise the kid as their own. He may honstly think that is what you want, and think his help would be unwelcome.
On the plus side, you are living a lifestyle I have always fantasized about. Lots of gay people never get to become parents, and you are becoming a parent without having to pay for sperm banks, with two potential co-parents (or at least a potential source of child support). I'm starting to wish I had had children.
You may have done this already, but in case you haven't...
Try talking to them. Try asking for help when you are tired. Reassure the man that you are interested in him. When the child is born, try to involve both as much in the kids life as you can. A fetus is an abstraction. A baby is an adorable commercial for parenting. Until then se if you can get either of them to go to doct'r appointments with you. (Studies show that is the best predictor of parental involvement).