I bought washable tempura paint from an art supply store, and it cost a bit less than the stuff marketed to kids in other stores. The art store sells tempura paints in powder form, so you can mix it yourself. I have limited myself to buying red, yellow, blue, black and white, and we make other colors by mixing these. I've read that adding 1 or 2 drops of dishsoap makes cleanup easier, but I haven't bothered added soap, and I haven't had any problems.
DD doesn't like painting on paper, but I have found that large pieces of scrap plywood make a great painting surface for toddlers. They stay put, and the wood surface takes globs of paints better than paper does. I just lean them against a wall between uses.