I second the idea of adding stevia....it is really yummy that way! Stevia gives some things a bit of a strange after-taste, IMO, but it's great in RRL tea and in every fruit-based concoction I've tried...not so great in coffee or chocolate flavored stuff, IMO.
Also, I just made up some RRL tea cubes! I've got them in the freezer waiting for when I go into labor, but I can't resist sneaking one here and there! LOL!
I made really strong RRL tea (I added a couple bags of Celestial Seasonings Mandarin Orange Spice, because it is one of my favorites!) and I added quite a bit of honey to it. I used honey instead of Stevia for this because honey provides extra, quick energy, which I will need during labor! Then I froze it in ice cube trays (I looked for little cocktail cube trays but couldn't find any...but with the honey, these don't freeze as solidly as ice cubes, so they'll be easy to crunch a bit anyway).
They are really good, and refreshing!
Also, if you are using RRL tea from anywhere but The Compleat Mother, you are missing out on a serious treat! The quality and taste of their RRL tea is MUCH better than any other! You may discover that you like it all by itself quite well!
As far as when to use RRL, my MW said it is totally okay throughout pregnancy, and beyond! No limits, no cautions. Drink as much as I want, and enjoy it!
I don't know about capsules...I've not seen it in capsules before (but I'm sure it's available!), but if you stick with a well-known, quality brand, I'd think you would be fine. There are wide variations among different herbalists, but IME, a good quality brand name will offer consistent quality across the board, if that makes sense.
Hope this helps!