I will say not all of the negatives I've listed are from my own experience, but I've heard them. And despite them all homeschooling is the best for us now; easier, better academics and social opportunities, all the good things definitely applied. But you asked for negatives, so here goes:
There are wonderful aspects to school that homeschooling can't replicate. That's the big negative for me. There is a different level of community at a good school. My daughter was at a Waldorf school and I taught there. I saw positive socialization there for both children and parents (lots of wonderful community events). However homeschooling is infinitely better than the awful school we tried after moving away from the waldorf school.
Socializing is much better than the bad school, academics are better, she was able to skip a grade with calvert without all the brouhaha such a suggestion would have caused at a school. Arguing with a bad teacher about your child's abilities can be frustrating; I must say, though, from experience, that having your child with a teacher you love, respect and utterly trust is fantastic.
A good relationship with teachers and classmates can be wonderful for children; to have a little life that is their own. This is hard to manage homeschooling especially if you have to be the social manager and chauffer as well. hschld kids have less privacy than they need, I think. Some children also simply learn better from teachers. I personally think all 4-6 year olds should be in a non-academic kindy/preschool whether they'll be homeschooled or not.
It is actually easier for me to get help with childcare now (I am a doula) because i know a lot of very flexible people and I don't have to ask someone to take her to school as well as take her in at 4am. But a block of time away from children is a godsend for a lot of parents, especially if you can work those hours.
It can be hard to be committed to homeschooling when you think that if your older children were in school it would be easier to pursue your career. It can be easy to get too wrapped up in it, esp at first; easy to burn out, or ride your kids too hard. I'd say a homeschooling mother *must* have her own interests; take a class out or by correspondence; hobby; work in or out of the home. I see a lot of hschled kids who have never learned not to interrupt mom, or treat her respectfully. ie, spoiled.
It can also be too easy to do the minimum and get lazy; all teachers, hschling or at school, can fall into this.
Homeschooling can get expensive. It can be done cheaply, but if you choose to enroll in a correspondence school, have lessons and classes (swimming, PE, music, art, tutoring, museum classes for homeschoolers) it adds up. For me it is still cheaper than private school, but it can be costly.
Socially for you, it can be nice; but a lot of lessons require a parent to stay and you can feel your life is ticking away in endless conversation about homeschooling with other mothers when you'd rather be doing something else.
I would prefer to homeschool in a city than in the suburbs: resources are better and you don't have to drive everywhere. But I am a new yorker stuck in FL temporarily so this might not be the same for everyone.
External criticism, esp from family, can be hard to take and can undermine a healthy and productive homeschooling environment, because children will be aware of the disapproval.
Again, I am really happy to be homeschooling and the positives easily outweigh the negs, at least for now. We can list those if you'd like.
Good luck!