Both my brother and I did this, in slightly different ways, at about this age.
For me, the typical scenario was that I felt criticized and ran to my room to cry loudly and dramatically, showing how unfairly I'd been treated and hoping for an equally dramatic apology. I was trying to escape criticism or punishment by being already so heartbroken that they wouldn't kick me while I was down.
For my brother, the typical scenario was that either (a) like your son, he realized he'd done something wrong and set out to punish himself so it wouldn't be done to him, or (b) he failed to meet his own expectations (for example, he was trying to make a cardboard robot and it didn't work out) so he punished himself for that. He would bellow, "I'M GOING TO MY ROOM WITHOUT SUPPER!!!" (our parents never withheld meals!) and stomp into his room.
For both of us, part of the problem was witnessing our parents' high standards, not so much for us as for themselves. They would agonize over making the right decision (about what to buy, how to do something, etc.) and then if they felt they'd chosen wrong, they'd go on and on about all the trouble it caused and all the reasons they should've known better. I have very mixed feelings about this: thinking out loud is GOOD for showing kids how we reason through decisions and cope with the consequences of our actions, and it's good to know that adults aren't perfect and don't always know what's best...but modeling excessive self-criticism is not such a good thing.
In part, this was a stage we each went through that resolved itself. In part, my parents helped by talking with us WHEN WE WERE CALM (not in the moment) about how the big, bad feelings of having made the wrong choice can be very overwhelming and how to calm the self-punishing urges and direct your focus toward solving the problem. It was really helpful to have my parents explain some of the self-calming strategies they used themselves, which were less evident to me than their ruminating over what had gone wrong--I'd thought that the only reason they simply grumbled about these things rather than shrieking and crying and slamming doors was that they'd outgrown that behavior, so it was interesting to learn that there are ways to talk yourself down from throwing a fit.
So, it's worth looking at how you handle it when YOU do something wrong, to see what you're modeling and what self-calming strategies may be hidden from your child.
Other ideas:
I think One Girl is right that he's protecting his feelings and sense of self-control by imposing the time-out on himself rather than letting you do it. I like the idea of talking (at a calm time) about solutions for specific problems that are coming up repeatedly.
My mom used to do that, and I think it's a good way to give the kid more control over his own behavior and help each of you understand the other's point of view.
He may also be testing whether your love for him (in the form of willingness to have him around) is conditional on his behavior. You could try distracting him from the unwanted behavior with a sudden fit of hugging, and see if that helps.

In general, more redirection might help. My son is only 4, but I've found that often when he continues an unwanted behavior, it's because I've only said "don't" without giving him anything else to do. It can be hard to think of something to redirect him to, when I'm already trying to do a zillion things myself and just want him out of my hair, but it often pays off!
For times when you're telling him TO do something and he isn't, try to resist repeating yourself too soon. This is really hard for me because we're often in a hurry, but again, it pays off when I do it! Say, "Please brush your teeth now." one time and then move on, doing your best to give every impression that you expect he will do it.
Also, try saying, "When...then..." First use it to motivate him to do what you're saying: "When you've brushed your teeth, then I'll read to you." Then use it to stop whatever he's doing instead of what you told him to do: "When you've brushed your teeth, then you can finish your Lego building."
For persistent rowdy behavior, my mom used to send my brother to run around the house 5 times. It was an excellent distraction as well as helping to burn off his excess energy.