I did a one-time babysitting stint for a mom who parents quite a bit differently from me. For instance, her baby has scheduled nap-times where he fusses a bit and soothes himself off to sleep; he's also expected to self-soothe if he wakes up before he's had a 2-hour nap. She only wanted me to intervene if the fussing just went on and on, longer than a few minutes.
Well, it was upsetting to my then 6yo dd to hear this baby fussing for a few minutes (through the baby monitor) before falling to sleep. She knows I never leave her baby sister to fuss at all -- and we don't even have a baby monitor; I usually hold her on me or lie down with her in the bed when she naps. And I never left my oldest to fuss, either.
It seemed to kind of shock my dd that I didn't just rush in to pick up this little boy, even after I explained I was following his mother's instructions (which dd had also heard), and not doing what I myself would normally do. In my case, I just decided not to babysit for this mom anymore: I didn't like acting inconsistently with my beliefs, especially in front of my dd.
If the mom had been a closer friend, or if I'd wanted to keep babysitting this child, I probably would have explained my concerns to see if we could work something out. But in this particular situation, I didn't feel it was worth it to bring up the issue. She seemed pretty strong in her belief that she was doing the right thing; also since she paid pretty well, she probably felt that whoever she paid should just do as she said.
Since you have children of your own, you might not want to watch a child whose mother wants you to give care that differs from what you believe in. Well, and even if you DIDN'T have your own kids watching, you still might not want to, just because you want to be consistent as a person.
If you really want to continue the babysitting, I'd talk with your friend about your concerns and say it really goes against your grain to NOT hold a fussy baby. In MY case, it even goes against my grain to not just continuously hold and wear a child this small. Then it'll be up to your friend to decide if she wants to keep you as babysitter, or find a more detached caregiver (which honestly shouldn't be too hard to do).