Another important consideration is what kind of community support can you get for German? Are you near any German schools? Even weekend schools?
Now, with lots of time, you might want to research this. The point would be if there is a move in your future, keep in mind locating yourselves close to where you could reasonably take advantage. Most young families move at least once before their children hit school age.
Speaking a non-native language, one that is not spoken in the community takes commitment but there are plenty who do just that. It's very trendy right now to do that with English in Asia. There are complaints that totally local children are entering school not knowing the community language! So you know it's working!!
Get your families on board with this project. Even if they're far away. Some of us get grief for using our native languages with our children from our in-laws, even parents. "Oh you're confusing him!" "I can't talk to my own grandchild!" and any other bilingual myth they can haul out... Head this off by speaking to them first and keeping them in the loop. Assure them that a child growing up in America, with at least one parent using English with them WILL speak English! Sounds like a no-brainer but it has cropped up as a serious problem with some families.
Does your dh know any German? No, he doesn't have to take a crash course but he will learn listening to you. My dh can now understand English quite well. But if he has at least some German, he can reinforce it with picture books, games and "testing" ("What does mommy call...?")
You will hear from nay-sayers. They will claim that using a non-native language will impede communication with your child. They do have a point but in your case, your child will learn your native language, English so that if you need to do a heart-to-heart in certain subjects, years in the future, you can revert to English if necessary. The pitfall is when a non-native parent uses the community language, cutting off the opportunity for the child to learn the parent's native language and the ability for them to communicate in it for good. That is definitely not your case so you're at least warned that you might hear this argument.
Meanwhile, before the baby arrives, try to get into "German mode". Slap some German decorations up around your digs. Watch some videos. Read some books (or at least magazines!) to get going on the German. You also might want to look on the internet for baby vocabulary, which I doubt you have unless you worked with German children in the past. Learn some children's songs, etc.
Good luck with both the birth and the language project!