What I've learned about having mice:
My cat sucks at catching them (he has no teeth, short claws and no interest). Works for some though.
Glue traps are a slow, painful, starving or leg-chewing-off death. I don't recommend this method.
Snap traps are a quick and relatively painless death IF the mouse is smart enough to stick his head in there. I found one trap with a limb stuck in it bear-trap style dragged across the garage. I felt horrible. But I recommend these anyway. You can't successfully reuse them... the mice smell death on them and stay away. The cheap, disposable ones are fine.
Havahart traps are a wonderful idea in the summer. In the winter the mice will come back inside, freeze outside (since they have no home) get eaten or even starve. In the summer you may be successful if you drive them far away, but they're smart and will find their way back to warmth and free food.
Also, if you don't check often in the winter, you'll find a frozen little mouse in the trap, and still feel lousy (it'll freeze or dehydrate in a short period of time). Yep, it happened to me.
Poisons are horrid... you don't want the mice dying in your walls, even if the box says they'll go in search of water outside. That is stinky and gross.
I recommend the snap traps, baited with peanut butter and left where little fingers can't get them (in locked cabinets in the back wherever you find the critters), in the garage, or removed in the morning. The kids don't have to know about them.
I love mice, had pet mice (well, all sorts of little critters) and hate to kill them, but having them in the walls is a health and fire hazard. It's hard to kill those cute little buggers, but a necessary evil, I guess.