From a Catholic perspective, a marriage between a Catholic and a Mormon would be considered to be disparity of cult since Mormons are not considered to be baptized Christians. For your BIL, such a marriage is not impossible, just difficult. There would have to be a dispensation to marry someone who is not considered to have a valid baptism, and the wedding would have to take place in a Catholic Church before a Catholic priest or deacon, or there would need to be a dispensation. He would be encouraged to raise the children Catholic. In previous times, the non-Catholic spouse would have to sign a statement that the children would be raised Catholic, or they would not be allowed to be married, and if they went ahead, he would be excommunicated. This is no longer the case, but that doesn't mean that the Catholic church desires any less that the children be raised Catholic.
From an LDS perspective, there are 3 levels of heaven, and in order to reach the highest level of heaven, one must be married, and the marriage must be sealed in the temple "for time and eternity." Basically, for her, if they never get married in the temple (which can be done any time, including after death, though it it was performed after death, he would still need to accept it), she will never make it to the highest level of heaven. The other levels of heaven aren't that bad (the bottom level is similar to Earth), but I've never known a Mormon who wishes to settle for one of the lower levels of heaven. She would need to be hoping and waiting for him to convert so that they could have a temple marriage so that their family could be united forever, and she would lose him if he did not marry her in the temple. It's questionable if this could work out from an LDS perspective. He could theoretically practice his faith throughout his life and she hers, and somebody could perform an LDS baptism for him (against Catholic teaching) and a temple marriage for them after he dies, and then the Mormons could hope that they both eventually make it to the highest level of heaven, but that's not really a solution that makes Catholics or Mormons terribly comfortable.
On both sides, there's some difficulties with the role of spouses within marriage and plans on how to raise the children, but I would say those are less serious than the difficulties associated with the different understandings of marriage. Also, Catholics believe that marriage is for this life, and the relationship that we have in the next life will be perfect, but will be something other than marriage. Mormons believe that marriages that do not occur in the temple are for this life only, and it is questionable if there will be any relationship in the next life. Marriages that are blessed in the temple are for time and eternity, and through them, there is a potential for reaching the highest level of heaven and becoming like God, becoming the god of another world, creating spirit children to populate it. It is this understanding of God that is what keeps Mormons from being considered Christians according to the Catholic Church (though they consider themselves to be Christians), but this is what he would be expected to eventually take part in.
As far as his kids learning things that he can't know, I wouldn't consider the LDS beliefs to be as closed as most people think they are. The missionaries have an order that they like to talk about things in so that people can accept the more basic beliefs before they move on to the more complicated beliefs, but there is very little that people who are not LDS just cannot know. Most of that revolves around the temple ordinances. What happens in the temple is considered to be very sacred and is not shared with non-Mormons, and he would not be allowed to even enter the temple. His wife and children would not be allowed to describe certain things in the temple to him, but in general, the beliefs can be learned by non-Mormons. I spent an hour or 2 a day, 5 days a week for the better part of 2 years with a devout and educated LDS friend, mostly talking about faith to get a reasonable handle on many of the beliefs of the LDS, and I'm sure there is so much more I don't know.