I think you should trust your gut on these type of things. If your gut says baby is healthy and don't do it, chances are you're right. And if you feel like you wouldn't change the way you're doing things regardless of the test results, it doesn't matter much anyway.
I did do the test, but it was suggested due to prior issues I have and some known genetic issues that run in my family. And I was really pleased with it, but if I wasn't at an increased risk I would've skipped it.