Adding to what honey-lilac mentioned--in my city, you ALSO need to get a city business license. All in all, it does seem overwhelming, but once you do it, it's done (till next year)!
The hard part for me is paying the monthly taxes!! State and city, separately. That's been the hardest for me to keep up with. But I finally got November and Dec 2009 paid, and it's a very easy, online process here. So, I'm a bit less overwhelmed. Filing my personal taxes as a sole proprietor...ummm, hopefully Turbo Tax Business will guide me through, otherwise I'm clueless.
oh, btw, I just started my LLC in November 2009.