Some of my must-haves:
Cloth Diapers: Half a dozen kissaluv newborn fitteds, a dozen mother-ease one-size diapers, at least 36 prefolds, 6 or so Thirsty covers and a couple Disana wool covers. A 13 gallon kitchen garbage pail with two PUL liners. Lots of cloth wipes and a few good waterproof changing pads (PUL covered with cotton, for example). If you're using cloth and out-and-about, you'll want a couple wet bags that fit inside a diaper bag.
Nursing: A tube of lansinoh lanolin, a nursing pillow (I used a Boppy) and some breast pads (though I had low supply and never needed them, you might), a phone number of good local lactation consultant - just in case. A safe nursing necklace is fun for baby.
Car Seat: I like the Britax Marathon 70 convertible.
Crib: I side-car'd the crib next to the bed so I needed to get it set up well ahead of time to make it even with the mattress, etc. The one thing I splurged on was a Naturepedic mattress. If baby is co-sleeping in bed with you, some wool puddle pads or a safe waterproof mattress cover (Naturepedic makes one)
Clothing: I liked sleep sacks at night because they made changes easier, extenders for one piece clothing gave us extra wear-time. My son didn't really care to be swaddled.
Accessories: A baby swing, baby carrier (which brand is hugely a personal preference but I'd not recommend a sling type carriers), a play yard and jumper (I didn't use these much but a few times they were very handy - this depends on lifestyle, etc.)
Health: A safe baby toothbrush, baby nail clippers, a good thermometer, a nose bulb syringe
Toys: As a baby, my son's favorites were the visually (black/white/red contrast colors) and tactile type of toys. Lamaze has a lot of cute ones. Get a few things you feel comfortable with baby chewing on - cotton toys are nice. I honestly think many of the things on the market today are almost over-stimulating. We had one of those playmats and looking back at videos of my son laying on it with so much going on - he was totally overstimulated. :D
Food: Stock up on freezer meals!
You'll have plenty of time to get a lot of the other things - high chair, any additional feeding supplies, etc. I wish with my first I hadn't bought as much as I did upfront because it took me a little bit to adjust and figure out what I liked and what worked. For instance, we bought a stroller and baby car seat. The baby car seat only got a little use and the stroller, zero.