I live with a whole family of packrats.
DH? No, that's rude. I'd just point out the clutter to him, and ask him to get rid of it or put it away. I'd beg and plead and then eventually put my foot down and insist.
The kids? Mine are still young, and yeah, I do get rid of stuff without asking them. Not cherished toys, not anything they still love and play with, but we'd be overrun if I didn't throw away stuff sometimes.
They could have forgotten about it for a month, but if I show it to them and ask can I throw it away, all of a sudden it's their most treasured possession. I'm talking about sticks and pinecones and random scraps of paper and cheapie plastic crap doohickeys from goody bags and stuff like that. Not to mention long-outgrown baby toys. I think in those cases it's kinder to just throw the stuff away on the sly, rather than putting the child through the emotional wrench of knowing it's being thrown out.
(For example, DD1 had these glittery pipe cleaners she arranged into a bird's nest, and played with for about a week. Then it fell behind her dresser, and she forgot about it. At least two months later, I found it there, all covered in dust bunnies and smashed. If I showed it to her, she would have cried about it being smashed, insisted she HAD to keep it, and then promptly dropped it somewhere else and forgotten about it again. We'd repeat this scenario at least twice daily, every single day, for SOME random item. Multiply that by three kids, and you have a house flooded in junk.)
I would never throw away anything if the child still remembered s/he owned it, and was still playing with it. Even the pinecones and scraps of paper. I wait until they've clearly moved on.
I do weed out outgrown clothes without asking, too, unless the clothing item is clearly cherished, like DD1's velour pants that she still wears around the house even though they're 3Ts and she's a 6X. I'm thinking those will make nice cutoff shorts; good thing DD is skinny.