Congratulations on your new little one! I think it's great that you want to make a quilt for them.
The pp's are absolutely correct- quilting is totally addictive. My first quilt ever was a baby quilt for a friend. There are lots of
different things you could do but the first thing that comes to mind is for you to look at some of Alex Anderson's books. She is a prominent quilter who has several books with clear directions even for hand sewing. Another book you might try is from Eleanor Burns- she has a series of books called "Quilt in a Day", but I think hers are more for machine sewing. The other place you can look is on Martha Stewart's website. She has several baby quilt projects and at least one is for hand sewing. Just search for baby quilt or quilt and you can print out the directions.
The easiest way to start would be to pick what size you want, then use a whole piece of fabric that size for the front and back(this would be called a whole cloth quilt), then layer it with batting inside and tie the layers together to quilt it. Don't worry about not having a machine, you can do it without one! I think you can find the whole cloth project on Martha Stewart's website and that would be a good place to start. Then take the directions to your fabric store and tell them what you're trying to do.
Good luck! I hope any of this helps and you'll love looking back at the quilt you made for your little one.