My oldest (8) is unschooled and LOVES it. Loves it. Now, she does spend a lot of time reading and likes to do academic things but never with me leading, never with anything that "looks" like school. It's all her.
My 2nd (5) asked to go to school this fall. She's the kid who didn't want to go to the activities I'd plan . . .and I'm talkin' the zoo, arboretum, the park, etc. Well, let me tell you, she LOVES it. Interestingly, she is the same as 1st DD is academically (both read fluently by the time they were 4) but 2nd DD is not bored by school the way my first one was, or at least not yet. She is thriving on the routine and structure. Frankly, I do think you can have routine and structure in an unschooling world (and that you should, if that's what the child needs) but I personally could not give that to her for many reasons, but the biggest is my 9 month old. (Happy baby but takes up all my time when we're home, which is why I'd rather be out . . .which is what my 2nd DD did not like. If we were home, then I couldn't give her structure anyway.)
I have been feeling lately, however, that even though DD1 (and 3) are fine with no structure or routine that I need it. So, we are doing really rough plans . . .zoo day on Monday, library Tuesday, Wed. a class, etc. I am also finding little (cheap) activities like a chorus, peace group for kids, and local history program for children. These are all free or low cost. The other thing I did was start a local homeschool group . . .it's small (or at least the # of people that actually show up is), but I've met some great people. I am also going to start a Roots and Shoots club. It's $50 per year but you could get everyone to chip in.
So, I say, try to be proactive . . try your 2nd in school. There will be no harm done because you could take him out if you know it's not working. He will TELL you if it's not.