Yes, I only do my decluttering work when I have the house to myself.
Mostly the issue is DD. The second I start making piles of stuff, she's into them. She'll make a mess, not to mention just annoy me with all her questions. Plus it's a task I like to really focus on, and I don't want to entertain someone else while I'm doing it.
So, in the warm season I sometimes beg DH to take her to the park or something so I can get something done.
Yes, some of the reason is so I can declutter some things against DD's wishes. She is a major packrat and doesn't like to see anything of hers given away - not even a stained shirt two sizes too small. I am very careful about decluttering toys, and so far have not made anything disappear that she noticed, with the sole exception of one stupid little stuffed dragon that she never played with but she once saw in the background of a photograph ("Mom, where is that dragon?" "Uh, we'll have to look for it..." case dropped, thankfully).
I really like to Freecycle and HATE to throw things away. That causes problems in my family, unfortunately. DH hates Freecycle in numerous ways, and would rather just load up the station wagon for a few trips to the landfill. The idea makes me so nauseous that I would rather just keep the junk. The problem is that most of the stuff I want to get rid of is truly perfectly good. My mother buys us way too much crap, so having things new in box or with tags still on them - I would feel like vomiting if I were to simply toss them in the landfill. So much waste. But DH hates humanity and does not want people on our porch or property, and he hates having clutter sit on the porch for a few days while I work through giving it away (something else I can't understand at all - our whole house is cluttered, it's not like I'm spoiling an otherwise pristine porch with a few bags). And he reads about people getting murdered over their stuff. Sigh. I've been getting my way over the last year, though, thankfully.
And DD, it's just that if she sees something sitting out and waiting for someone to pick it up, it can turn into a mess of questions and so on. I finally got smart enough to store a big, opaque plastic tote bin out there, and I bag up the stuff and stick it in there and tell the freecyclers to look in there. So DD is used to the box being there, never wonders about it, and has no idea things are going in and out.