My DS was a miserable infant. He cried for four months, and it was awful. What we've figured out, in hindsight, is that he wasn't sleeping enough, especially during the day. He was horribly overtired, almost all the time, because he could only sleep on my body, and I had another baby (his twin) and a two year old. So he wasn't getting enough sleep. The other thing we realized is that he's just an intense child--- even now, he's my most needy child. He's sensitive, he's easily startled, he needs a lot more physical contact and affection and attention, and things just upset him more. I think we were seeing that intensity manifest itself right from the beginning.
I would think about sleep first of all, honestly. So often newborn fussiness seems to be related to an overwhelmed, overtired infant. Have you tried a sleep routine? That's not a schedule, and it's not cry-it-out or anything-- it just means that 60 to 90 minutes after baby wakes, you start trying for another nap-- however you normally get baby to sleep. You're trying for no more than 90 minutes awake at a time, with one longer period of about two hours, late in the day, especially as baby nears four months old. It made a huge difference for us, once we figured it out.
Another thing that helped us was reducing daytime stimulation. Have you tried wearing baby early in the day, before the fussing even starts? It can really help, especially if you're on your feet and moving around during that time. Long, steady walks outdoors, in a carrier, early in the day, was our best solution. The more they sleep, and spend soothed, early in the day, the less likely they are to finish the day a miserable mess. In fact, the outdoors was our solution to most fussiness-- one of my best strategies was to take DS for long walks after dinner, in the fresh air, before bedtime. He calmed down a lot outdoors.
And

because living with a fussy infant can be so hard. Hang in there-- it does get better quickly. The peak of infant fussiness is about six weeks, and for non-colicky babies it declines quickly after that. For "colicky" babies, the situation seems to improve late in the third month. So though it seems like an eternity now, it really does go so fast.