I would think a couple of ounces a day would be fine, for an older baby. We started offering water with solids around 7 months' or so, and she always has a few sips with her "meals" but we're still mostly breastfeeding.
The worry I think is that if you give them too much water not only does it fill them up without the nourishment of breastmilk, but it can make their electrolytes go all wonky. That's why it's best not to give it to a baby under 6 months or so at all, at least according to what I've read.
We found a good teething trick though was to use a newborn nipple on a bottle and fill it with very cold water. The baby chews on the nipple as a teether, and it helps. She doesn't get much water from sucking but the cold feels good on her teeth. She doesn't drink from a bottle anymore beyond the chewing thing on occasion.
At mealtime though she either drinks from a sippy cup or a regular cup, and probably takes about 1-2 ounces, if that, per seating. (So a total of probably 3-5 ounces a day?)