Don't be shy about sharing your mother tongue with your children. And if your relationship with DH is rock solid, there's no reason for either of you to have any insecurity if you or he is speaking a language the other doesn't understand.
We had this discussion on the multicultural families forum of thebabycenter.com and it was mainly some moms feeling self-conscious about speaking to their kids at the playground in a foreign tongue that the locals wouldn't understand. Yet at our playground, I witness on a regular basis Japanese moms talking to their multicultural children in Japanese, and I think it's a form of pride and forging links with grandparents that might not be able to communicate with the grandkids in French/English/Dutch/German etc
We use OPOL and though I wish my DH could converse in Cantonese with me, he has been *very* supportive that I use my mother tongue with the kids, even if he can't understand Chinese (he understands the basics like sleep, eat, diaper change via the context). I get the oldest to play translator for DH every now and then.
I adore my grandparents and wish they didn't live so far away so that I could see them more often. They don't speak English or French, so I do feel compelled for my kids to learn Chinese so they can have meaningful interactions with my extended family abroad. Not to mention that 1.1 billion people already speak *that* language and knowing it would be an asset for work, studies or travel in the future! I do feel as a parent, the best I can give my children are a good sense of their roots and wings to fly and pursue their dreams, realizing their fullest potential.
Good luck whatever you choose to do!