There is a lot more involved in having a home-based food business than most people realize. Here's the basics that I remember, but it's not an exhaustive list.
1) You need to find out what the regulations are from your local health department regarding selling home-produced foods. Some cities/counties allow home-produced goods as long as they are clearly labeled as home-produced. Some require the labeling and an inspection of your home. Some do not allow any home produced goods and require everything to be prepared in an inspected commercial kitchen. Rental space in commercial kitchens is hard to come by.
2) You need a business license, sales tax permit, and food handlers permit (or higher food safety certification--some require you to do a food safety manager test which is much more extensive and expensive).
3) You need insurance for your business so that on the off-chance that you cause illness/injury to someone with your food items, you will be protected. Most commercial kitchens require proof of insurance before you can use them. Many cities require proof of insurance before they will issue a business license.
4) The labeling requirements can be pretty strict. The exact requirements vary from place to place, but if I remember correctly, the requirements here were that every item had to be labeled with the business name, business contact information, expiration date, weight, nutrition facts (in USDA approved format), ingredients (in USDA approved format), and allergens (again, in USDA approved format).
5) It can be difficult (but not impossible) to turn a profit selling food. People get used to paying Wal-Mart prices and aren't always willing to pay as much as it costs for you to make a profit. When factoring prices, remember to include all the licenses and such above, ingredients, utilities, packaging, delivery, etc.
6) If you plan to sell things online or ship them across state borders, there are additional rules.
As I mentioned, many of the requirements vary drastically based on where you live. The best bet is to get in touch with the health department or whoever controls such things in your area. Also, sometimes there are small business groups/organizations that you can join for help.
I was planning to start a home-based candy business, but ended up bowing out and pursuing a different path (which I know realize was a wise decision for me). While I was researching it, I became friends with a local lady who sold caramel apples from her home. She didn't do big business--she sold 5 or so apples per week. She had not followed all the guidelines above. A vindictive neighbor reported her to the health department, and she was fined $4500. It wasn't pretty.