My son is 4 and you could be describing him perfectly. We honestly don't push the issue and just let things happen naturally. This year he is old enough for Religious school so he has to go there, and he likes gymnastics but will only talk and interact with his teacher. Other than that, I let him play however he wants. I am concerned about kindy next year, but that will come in its own time and I am determined, ok well mostly determined, to just go with the flow. If he is having an extremely hard time with it, then we will have to look at other options. Honestly none of my kids had play groups, none of them had social engagements until they asked for them. If the activity became too stressful, then I had to remind myself that this was something THEY wanted to do, not something I wanted them to do. Really, reminding myself that a child will know what they want and pushing them will make them pull back helps. I also can't get too invested in THEIR activities, those are theirs to make their own. Just relax, maybe he'll come out of it, or maybe he'll be more like I was. I didn't like crowds or large groups, I didn't get a friend until I was 8 and even then it was 1 friend. I have always found 1 friend I can talk to and be around, any more than that and I feel lost in the crowd. i still won't go shopping on Black Friday because I can't stand it. But, you know what, unlike you I enjoyed my life like that and I wouldn't change it for the world.
I think the most important aspect is that you are paying attention. If he is miserable you will know it, then help with that situation. Right now, he isn't miserable playing by himself, he's miserable trying to fit into a group. Just deal with who he is, and make sure he knows that you are there to support him regardless.
Wife to dh since 1999, stepmom to dss (13 yrs. old), mom to ds (9 yrs. old), dd (7 yrs. old), and ds (4 yrs. old)