We lived in Vicksburg for almost a year and a half, and we enjoyed it. Our daughter is not school-age, so I can't say much about the schools (they are probably better in Clinton)....but if you're looking for a good, quiet apartment complex I highly recommend Bradford Ridge Apts. You won't find anything about them on the internet, but they're well taken care of and very reasonable rent. It's located on Cain Ridge road off of hwy 61 S.
If you were looking to buy a house, I would look towards Clinton. Vicksburg has some nice areas, but they are jammed up against the not-so-nice areas...and that was a concern for us as far as long term home values are concerned.
There is a midwife in town, Velma McAllister, she could probably point you in the right direction about doula info.