Water only! Especially if cord stump is still attached. We gave our babe sponge baths with water only until then, and once the stump fell off, we only gave her true baths like once a week (with her in the tub with me)? Sponge baths daily though.
Now that she's older (almost one year) we still do sponge baths, but full on baths maybe two times a week (depending on messy dipes, etc.). We use the Earths Best/Jason lavender organic baby wash/shampoo. I love it. It's tear-free and smells nice and she's never had a problem with it.
I never use soap on her face or ears though as it's too difficult to rinse without risking water in her ears and it's never been a problem to just use a damp cloth there.
If you encounter cradle cap, don't use any soap if possible (soap in any form dries skin) -- but you can use a soft bristle toothbrush to gently scrub the scalp. The olive oil trick is a good one too!