No, it wouldn't be irresponsible. It sounds like you would have guilt about though, and THAT isn't a good thing. If you want another child and can provide all the love, security, etc. that s/he would need, then I say go for it. Millions of children grow and thrive on formula alone.
But, I will say that if feeding your child breastmilk means a lot to you, you will be more prepared this time around to try to tackle your discomfort with it. Maybe talking to a some kind of professional? I'm not sure if pain was an issue for you, but it was for me. I almost stopped nursing because of it (and because, like you, I'm not a touchy person. I was always so uncomfortable and anxious at first). Most of that went aways by 2 -3 months...maybe it might for you next time. Also, you could always try pumping again but this time preparing yourself well for it. I don't know why pumping didn't work for you the first time, but maybe setting your mind to it really early on in your pregnancy might increase your chances of sticking with it.
With that said...if you don't want to nurse and you feel like you've exhausted all avenues that might change your mind, then please don't lose any sleep over it. I don't love nursing either, so I know where you're coming from.