Okay so I'm going to be the bad guy here, but we had a semi-feral cat that we adopted many years ago (it was actually DH's cat that he brought into the marriage). She was literally found in the woods near his house as a kitten, abandoned by her mother.
She was always very unpredictable and semi-violent, even with two non-aggressive adults who just wanted to pat her gently. The final straw was when I was seven months pregnant with DD. I brought home some sacks of groceries and she was sniffing them. I offered her the back of my hand to see if she would let me pat her. She struck out and scratched me very deeply, with no provocation. We took her to the Humane Society the next day.
If she acted this way with two adults who understood her behavior and still cared for her, a baby was out of the question.
I guess my story is just to say that if you have concerns about your cat's temperament now (which it doesn't sound like you do), consider them carefully!
I called the Humane Society the next day because one of our friends said she would've taken the cat, and she had already been adopted by another family. So, that helped my broken heart a little.