Baby boy's not due for 3 weeks, but DH is 30 years old and has never had a single issue. I've personally known a bunch of intact guys and none of them have ever had an issue.
On the flip side, I've known QUITE a few circ'd adult men who have problems as a result (incomplete circ's, scarring, re-adhesion issues).