That is a really big difference of opinion. It is unlikely either one of you will change your mind. You need to really do some soul searching right now, and decide if you CAN only have one kid, or if down the road, it will be too much to bear.
If my husband had said no more after one, to be honest, I probably would divorce him. I love him, but it is extremely important to me to have several children