Yes, I get this too. Even when my kids were in school, I was excited to have them home all the time for summers and vacations.
I think, though, I know one reason why people say this: It really is harder to have schooled kids home for vacations than it is to have homeschooled kids home all the time. It's like the schooled kids are having mini-deschooling experiences: they and the parents really don't know what to do with themselves when they are home together all day. So, the kids act out more, and the parents - not having developed the tools for dealing with bored kids at home - feel frustrated and at a loss.
I have several friends who ask me this in the summer: How can you do this all the time, when I can hardly even get through a summer with them? And I explain to them - In the summer, your kids are deschooling. My kids have already deschooled, so we don't go through all that boredom, and acting out stuff anymore. Simple!