Why would you want to?
You gave it to him, its his comfort - why would you want to take that away from him?
He won't still have it at 20 (and what if he did? - I know plenty of people that still suck their thumb and are in their 30's! lmao), it won't harm his speech and it won't mess up his teeth (well - as long as its not a cheap one and orthodontic, but most are orthodontic these days anyhow).
Whats your comfort? What if your husband/partner decided that he felt you had enough of that and took it away from you?
You don't need to clean them - thats a bit overkill. Any toddler should be eating a pound of dirt a day lmao...But seriously, more crap goes into a toddlers mouth than you might know of. Immune system building - they live. Have a spare, wash it later if you are out in public perhaps? There are ways around it. I have been there -I know they can be a bit of a pain but then I remind myself that at the end of the day, I gave it to my son. (of course a clip can also help prevent it from falling on public floors - I have never had one lost or fall on the floor when we started using the clip)
You can get clips
for them (they make all sorts - specific ones for your brand of soother, and even nice wooden ones! lol) that keep them attached and are safe - instant searching problem solved. They can't go far then if they fall out of their mouths at night.
Most children I know do give them up eventually. Its tough being a toddler. Then you get those horrid molars coming through and they can add extra comfort then. My son gave his up, on his own, no coercion from me, shortly after he got all his teeth. So he still needed it until then.
This is something I am pretty darn passionate about - having been the child who clearly remembers being made to stop sucking her fingers in all sorts of ways until it finally worked and I hated my parents for that - and though hate is not a pretty strong word for it, I still feel pretty upset about it. It was MY comfort - it caused NO problems for anyone. It was not their right to take away from me. Once its gone - you can NEVER get that kind of comfort back.
So something to think about.