The biggest problem with no Dairy and mainly Breastmilk is the lower fat and protein content of breastmilk. We went through this with my DD (and BM was the base of her diet until she was 2). You can add in more protein with meats, soy, fish, legumes, quinoa, etc. You can add yummy fats with avocado, oils, coconut, etc. Coconut milk ice cream has tons of yummy calories.
If they are really concerned, ask them to check her Albumin and pre-albumin levels (which determine malnutrition). Our dietician had us keep a 3 day diary of everything DD ate (or we tubed), then we went through, counted calories & nutrients, and made adjustments. In her case, she just needed more fats. I wouls look at her intake and make sure it is sufficient before I started down the elimination diet route.