Consider this: when you put your child in timeout until he stops crying, you are giving him (at least) three messages: 1. your impulse is wrong 2. your emotions are wrong. 3. And you will be isolated until you can stop being yourself.
Making a little kid say sorry when he doesn't really feel sorry (and probably it isn't yet in him developmentally to have the empathy required to feel sorry) is teaching him to LIE. AND that it is better to lie and not get in trouble, or get out of trouble, than it is to express himself honestly.
It is possible to teach your child right from wrong, without also teaching him that HE is wrong, and that lying is ok.
We try to do three things: 1. talk about what the problem is. 2. offer a similar alternative 3. validate/empathize with his feelings/impulse.
If DS does something wrong, let's say he throws food on the floor, first I might say, Oh, we don't throw food on the floor, would you pick it up, please? Sometimes he will and sometimes not. It isn't important to a two year old that the floor is clean. It is important to me, so if he doesn't pick it up, I do, and reiterate, we don't throw food on the floor. Then I offer an alternative. "We don't throw food on the floor, but we can throw the lids on the floor. They make a fun sound, too." (We keep a box of lids in a lower cupboard in the kitchen, which have been unending fun for DS since he could crawl. In fact, there is nothing in any of the cupboards that DS can reach that he isn't allowed to get into. They are just pots and pans, and they are great to bang on with spoons. So DS is totally allowed in any cupboard he wants. Why not?) Honestly, we almost never get farther than the alternative on any issue. But lets just say for the example's sake, that he didn't want to throw lids instead, that he broke down in a tantrum crying because he couldn't throw food on the floor. I wouldn't say, now go sit by yourself, with all your two year old powers of contemplation until you can figure out what you did wrong, control your emotions, and actually feel sorry about it. I would just sit with him while he cried, offer a hug if he wanted it, or to nurse if he wanted it, until he had finished expressing his frustration/sorrow/anger about not being allowed to throw food. And I would offer him some empathy/validation. "You really wanted to throw food on the floor. And I wouldn't let you. I see you feel upset."
But honestly, we don't have tantrums. We don't have crying fits. The terrible twos IMO is just a big parental misunderstanding, that the only way to teach your toddler is through "discipline". It is a misunderstanding that your child is wrong. Your kid wouldn't be wrong for wanting to throw food on the floor. He would just be experimenting. Maybe with gravity, maybe with what you will do, maybe with what is ok or not ok. You can empower him to find the answers to his questions, which are after all, part of his learning to be part of our society, without teaching him that HE is wrong. He isn't wrong. He is right, he is just looking for answers, that you can give him in a kind way.
My nephew does pretty much everything you described. It's the beginning of the terrible twos.
He likes pulling my mom's antique books off the shelves, stealing toys from his sister, opening kitchen cupboards...pretty much everything we don't let him do. We don't spank him or anything like that. He has a little chair that he loves sitting in, but if he gets in trouble it turns into his timeout chair. He has to sit there until he stops crying, and then we ask him if he's ready to say sorry to whoever he hurt or put him in timeout. If he starts crying and throwing a tantrum again, he has to stay in the chair. If he just looks at us, smiles, or starts getting out of his chair he has to go hug, kiss, and say I love you (he can't talk real well yet) to the person who he upset. If he gets up before he's allowed to, then he has to go back to his chair until he sits there.
It usually lasts less than 5 minutes, and he usually doesn't go back to it for awhile...meaning about an hour or two, unless he gets in trouble for that one a lot. The other thing we do is look at him sternly and tell him "Bryson Lee...No." And he'll glare, but generally stop.
Hope this gives you an idea.