How long has your daughter been off of dairy? Did your daughter also get sick frequently with colds and ear infections in her second year of life? Does your daughter have any sensory issues that prevent her from chewing and swallowing solid food effectively? Does it feel to you that her pediatrician is ruling out possible causes of her small size or does she have a specific concern she's testing for?
My son was intolerant of dairy and gluten. Between the ages of 15 months and 2 years he was sick every other week. As a result he was often not hungry as most people wouldn't be when they are chronically stuffy nosed and congested. He actually lost two pounds during this time.
On top of that my son was unable to chew and swallow food that did not have crunchy textures. He was able to swallow mashed or pureed food however. His sensory issues made eating feel less fun for him.
My son's pediatrician while concerned didn't call it "failure to thrive" although I think a doctor with less experience might have. I think for us, because she was an experienced mother and doctor, she recognized what was happening to my son, and she followed us closely for a few months to ensure that he regained his weight and began growing as he should.
I trusted her judgement and felt that she made the right call. She is a warm and caring doctor who listens to me and took the time to know me as a parent. I knew that when he was able to clear the residual effects of the foods he was intolerant of, stay healthy for a period of time and recover his strength, he would grow as he normally would. It didn't mean I took it for granted though and I was nervous. Being a parent for the first time and hearing a doctor say "failure to thrive" is frightening and you can feel blamed too. It is an integral part of being a mother that you nourish your child and they grow.
Now, I wouldn't say, don't have your child seen by specialists to rule out possible causes of her lack of growth. But, if you have doubts about the necessity for further tests, ask your pediatrician what the harm would be to wait a few months before doing so, to see if she begins growing larger with the changes you've implemented to her diet and removing an allergen from her diet? I suspect that you hesitate to be invasive if it isn't necessary because tests of this kind can be traumatic and can involve sedation so make sure you get good information before proceeding.
In my situation, I brought my son in to the Pediatrician for weekly weight checks, then biweekly and then monthly weight checks, for a period of time, maybe three months.....by the time he was 2 and a half he had gained back what he'd lost plus some....so things worked well for us.