His kids' attitude is not necessarily rational, or fair. It's certainly not selfless. But they are still kids, and kids - even nice ones - tend to be pretty self-centered and incapable of seeing the world with the same level of rationality, fairness and maturity that will hopefully acquire, by adulthood.
His kids have a relatively short time before graduating high school and (hopefully) going off to college or to otherwise begin independence. It is common for teenagers to feel insecure about a parent's attention and devotion to them - especially the parent they spend less time with - because they realize that, in a few years, both of them will learn to adjust to spending a lot less time together. "If my parent can adjust to that, am I really important to him, even now? If I don't find some way to prove that to myself now, maybe I'll never really know..."
Likewise, your BF realizes he only has a few short years left with his kids at home. IMO, his willingness to put that first should make you feel better about him and a future relationship/marriage with him, not worse. I realize you feel disappointed and that's understandable. But if you want long-term stability with him, not just fun now, his very inattention to you on the weeks his kids are with him is proof he has some good, loyal, devoted character traits!
I know it's not the same as a long-term relationship, but once I was set up on a blind date with a friend's brother. He was nice and so was the date, but I just wasn't attracted for whatever reason. Unfortunately, he was. I was supposed to drop by his office a few days later, for lunch. I had planned to find some excuse to give him, at lunch, for why I didn't think I could get more involved at that point, even though I did think he was very nice. He completely stood me up - didn't call my cell, leave a note, a message with co-workers, nothing! Finally, as I was leaving his office in embarrassment, a receptionist overheard who I had come to see. She mentioned that the nurse at his kid's school had called a half-hour ago and the guy had immediately jumped in his truck and left, without a word to anyone. The guy did call me, after his kid was at home asleep (his son lived with him). I have to tell you, I was really impressed that when he heard his kid was sick, he did not have a passing thought about the woman he was trying to impress - he just ran to the school and took care of his puking child. I went out with him again mainly because of that, to see if I could generate some spark for him, after seeing that side of his character.
Maybe try to look at things from that perspective? It's not about you personally, it's that he is a man who puts his kids first. That's a good man! You made a good choice. You're attracted to good things! And, most likely, once his kids are out on their own and he knows they're OK in the world, he's the type of man who will be capable of putting his spouse first, too. Waiting a few years until he's finished raising them is not so bad...unless you let yourself spend that time competing with his children, or giving him a hard time for ignoring you.