I take them all the time. Personally, I have enjoyed wearing them both (one on front in a mei tai, one on back in a backpack)...mine have always been happier that way when we are out, and it seemed like whenever I would have one in the seat, he would get fussy, or wake up and need to be held, and THEN things got hard trying to swap them around and what not. Now that they are getting bigger, I usually put one in the cart seat, and throw one on my back and have a sling underneath for the front, in case the one in the seat needs to nurse or be held.
I make sure I have my list organized roughly by section (produce, frozens, aisles, etc.), and move briskly. Usually I can make it out of there before someone gets fussy, but if not, I just end up nursing while waiting to be checked out.
When they were really little, sometimes I just went in the evenings by myself. It was nice to have time to myself and linger through the aisles. A good excuse to get out and have some alone time!
However, time with my husband is so precious right now that I'd rather get it done in the day so we can hang out at night!
Hope that helps, and don't be afraid! I remember thinking the same thing when I was pregnant with our twins. I thought I would not leave the house for 6 months! Little did I realize that I would NEED to get out at almost any cost!
The babies went to story time at the library, on nature center visits, hikes, meetings, whatever. The only down side is that I think half my town has seen me with my shirt half on!