I would suggest starting small. Just a few pots. The cost can add up quick, and then if something happens and you can't keep up or decide it's too much work you'll be disappointed. You can also find pots on CL or Freecycle sometimes though - which will help with the cost. Don't bother buying "soil" - try to find prepared compost instead, and then I mix compost with coconut fiber (for water retention - coconut is renewable, peat is not). If you have a friend who is experienced with gardening, ask them to come over and show you how to repot a plant. It's not difficult, but it's so much easier (for me) to learn by being shown than it is by words on a computer screen.
For a beginner, I would stay away from cilantro. It's really finicky. However mint is a great plant for a beginner, but it needs it's own pot, it can't share with anything else - and give it a bigger one than you think it needs - protect it in the winter and it'll be with you for years. For tomatoes and peppers, look for bush varieties rather than vines - something that grows on the smaller side... bells may not be a great choice space wise (I don't know, I've never succeeded with bells), but there are tons of different types of peppers to choose from. Another great one for a beginner is bush beans (green beans) - but again get a bush and not a vine for a patio garden.
Most of your greens don't like the heat of summer - they'll do well in spring and fall. Do some research on the right time to plant (particular species) for your area - your local nursery can probably help you with that. For summer growing you want your heat lovers like peppers, tomatoes, basil. Spring is great for peas, lettuces and tender herbs like dill and parsley.
I've also always loved growing strawberries in containers. You need to be sure you give them enough space and enough water, but they'll come back year after year if you protect them in the winter (I grow "wild" alpine strawberries, but most cultivated species only bear fruit for a couple years). Rosemary is another one that will come back year after year if you protect it.
And then find your toddler a watering can just his size, and let him help water. My guy LOVES his watering can, and always asks to water the flowers, even though we don't actually have any this year. You'll have to teach him not to hurt the plants, pull off the flowers, break branches, pull off the fruit before it's ripe, etc., but there's nothing you can't share with him if you stick with organic growing practices - including all the dirt he's liable to eat. 
Oh, and I wouldn't start by growing from seed. That's a quick way to frustration IME. The first year I always recommend you buy/get your plants already started. Once you know you can keep them alive, then you can try starting from seed next year.
Follow Mothering