That sounds really frustrating.
There's a big difference between picking someone apart and holding them accountable for the way they're treating you/other people, and based on what you've described it sounds like you're doing the latter. Obviously there's a right way and a wrong way to go about approaching those kinds of things, and the "right way" might look different depending on personality type, but in any partnership someone should be able to come to their spouse and tell them if they're hurt or if they're concerned about a certain behaviour. There's also a difference between a 'weakness' and a hurtful behaviour.
I don't know him so I can't speak to his motives, but the whole "I'm sure you have weaknesses but I'm not holding them against you" feels manipulative to me. He could genuinely feel attacked and is deflecting so that he can convince himself he doesn't need to be accountable for the "grumpiness" but regardless of intent, it's a poor reaction. Would he respond better to, "SO, there are so many things I love about you. I love that you ________ and that you ___________ and you're __________. One area that does hurt/frustrate/concern me, though, is ________. I'm sure that isn't your intent, but I was hoping we could talk about it." Some people might feel patronized with that kind of a lead in, but df and I both approach things that way sometimes so that each other won't be as likely to feel defensive so it can work with the right personality types (as long as it's sincere - if we're too angry to speak that way we try to shelf the conversation until we can be grown ups about it). Also, if it were us, "I'm sure I do have flaws, and if they're ever at a point where they are causing you hurt I would be more than open to talking with you about it. Right now, I would like to continue talking about behaviour x. If now isn't a good time, I would like to set aside a time when you'd be willing to." would probably be said at some point, to acknowledge but also to prevent the conversation from spiraling or rabbit trailing, since some people deliberately say those kinds of things to distract from themselves.
At the end of the day, though, he has to be the one to look internally and decide to start making changes. Individual counselling is probably a good idea for him to work through whatever causes him to have so much anxiety and/or help him find better ways to handle it but based on what you've said here I don't think he sounds like he's in a place where he'd be willing to take that step. You can only leave the door open, be loving and supportive without compromising your own boundaries and know when to walk away if need be.