To clarify my answer, let me first define terms:
swimming briefs: a one-piece swimming costume that resemble a pair of underpants - sometimes called a "speedo"
one-piece: a one-piece swimming costume that covers the torso and bottom; may have sleeves that cover the shoulders or thin shoulder straps that hold it up; may come down to cover the upper thighs (much like a unitard or short wetsuit) or include an attached skirt
bikini: a two-piece swimming costume that resembles women's underwear (bra & knickers)
tankini: a two-piece swimming costume that resembles a pair a of pants and a sleeveless top that may or may not fully cover the torso
swimming trunks or shorts: a one-piece swimming costume that resembles a pair of shorts
two-piece: a two-piece swimming costume that includes a top that fully covers the torso - may have full sleeves or shoulder straps - and a bottom that is either a swim skirt or swim trunks/shorts
rash guard: a top that provides full shoulder coverage with either short or long sleeves, and, due to the fabric, offers a high SPF
Now, if we're talking about babies in the 2-6 year range, I personally believe that a bikini is inappropriate. Why? Because it's highlighting breasts that aren't there, which sexualises a prepubescent child. If you don't agree with this belief, then answer me this: would you dress your 4 y/o in a bra? Because that's essentially what a bikini is: it's a bra made out of stretchy, quick drying material so that it can become wet and dry easily. As with a brassiere, there is no reason to wear one until one develops breasts.
After reading that, you might think that I'm an advocate for one-piece suits instead, but I'm not (at least, not for 2-6 y/o's). I agree that a one-piece is a pain for nappy changes and toilet use. And while we're on the subject, I don't believe in swim trunks for this age range, either. Babies at this age have absolutely no waist and trunks have a tendency to fall down once they're heavy with water (it doesn't take much at this age!). What's more, swim trunks tend to bunch up while one is swimming, and the netting often will give the wearer a wedgie - no fun.
I think the answer to the, "What swimming costume should my 2-6 y/o wear?" question is simple: swimming briefs. They don't sexualise children, they don't ride up or bunch, and they're easy to pull on and off. What to do about sun exposure, though? Well, for parents who feel sun exposure is a greater risk for their children, there are full length swimming costumes (to the wrists & ankles & neck) that protect against UV rays. This swimwear isn't as popular as the options mentioned above, but they do work and are an excellent solution to the risks of overexposure to UV rays. It's also pretty easy to put a rash guard on in addition to swimming briefs. And for those parents who don't feel sun exposure is as big of a risk for their children, a simple cover up, wide brimmed sunhat, and some sun screen will do the trick.