If your ex has indeed been charged with family violence against his new girlfriend AND CPS case re: her kids, that is a GAME CHANGER.
In other words, whether there is a court order or not, you need to notify CPS and ask for advice re: how to protect your son. Even though your son doesn't have specific allegations (by the way, when your son says "He's mean", do you ask him "how is he mean? What does he do? What don't you like?"? Those are important questions that may get more details from a 3 yr old), the fact that your ex has these allegations (and ESPECIALLY if he's been found guilty of any of them), that changes everything.
Contact CPS, and also contact the courts. You can basically just go to your local family court (wherever you think this court order sits) and ask how you find out the specifics. If there is an actual court order, you should have received that - did you get it but don't know where it is, you never got one, or you're not sure?
Contacting CPS is not so much because you have an allegation that he's has actually hurt your child (unless your child gives you more details), but instead it's to make sure you know all the things you can do to stop visitation until you are sure it's safe. It's about getting advice and finding out your options.
Lastly, even though some 3 yr olds can be overly dramatic about change and things like this, my experience (I work for CPS and have for almost 20 years) is that kids that young RARELY make up that kind of fear and anxiety. It's almost always real. And even if it's an "over reaction" to whatever your ex has actually done, it needs attention and your son needs help dealing with it.
I am all for doing everything possible and healthy to maintain strong bonds between children and the non-custodial parent, but NOT when it causes the child distress or the other parent has documented behaviors that very predictably could harm your child. In those cases you need to do all you can as the other parent to a) find out how bad (or not bad) your ex is treating your son and his new family, and then decide (with help and support if you want it) what is best for your son and figure out how to make it official.
If there is no formal court order, in most states you can just tell your ex that you're stopping visitation because you think it's having a very negative impact on your son. If he wants to try to go to court to establish visitation, you can bring in what you've learned about him (family violence charges, CPS charges) and that all matters in court. But if possible, it's best to just try to avoid sending your son to visit your ex until you can find out what is actualyl going re: his charges, CPS charges, and whether there is an actual visitation order in place or not. If you can make up reasons for not sending him while you find out, that may be best until you really know where you stand. He's likely to get upset if you tell him you're stopping it because you think it's really bad for your son, so you want to be ready and sure of your Plan A and your back up plan.
But once you know what your ex has done (to your son, his girlfriend, or her kids), what the status of those allegations are, and what your rights are, you do need to make an official stand and communicate it honestly to your ex.
If your ex is violent, you also need to think about support for yourself. What will you do if he has a bad reaction? How will your stay safe and keep your son safe? Does your ex have pick up rights at your son's school/daycare? If you end up not wanting himt o contact your son, make sure all the people your son stays with know what is going on and what to do if ex shows up.
Best of luck, please please let us know what you find out and what you decide to do!