And yet, there are numerous scholars who not only disagree with your view.. .but have for centuries. That's one of the beauties of Islam--that there can be two different opinions, both based on itjihad, both having merit... and we can use our own reasoning to decide what to follow.
Remember not all innovation is bad in Islam. There are both good and bad practices. From Sahih Muslim, "He who innaugurates a good practice (sanna fil-Islam sunnatun hasana) in Islam earns the reward for it, and for all who perform it after him, without diminishing their own rewards in the least."
Ibn Kathir wrote about celebrating the Mawlid.
Imam Al Suyati wrote about it being good.
Ibn Tamiyaa wrote about it as well being about "love and veneration of the Prophet."
And it goes on and on and on.... you can read more here: http://www.slideshare.net/johnycage45/mawlid-new
They also mention accounts (see pp. 24-25) of very early celebrations of the Mawlid. It is in Ibn Jubayr's "Travels" (540-614). It is in the 7th century historians Abul' Abazz al-'Azafi and his son's work. It is in the 8th century work of Ibn Battuta.
It is celebrated by millions and millions of Muslims--and it is not viewed as bad or against Islam by millions and millions. If their intention is to remember and love their Prophet, do you think Allah(swt) judges all of them negatively? Alhamdullilhah, that I believe in Ar-Rahman Ar-Rahim... a God of Mercy and Compassion. A God who loves and honors good intentions.
I'll also add... that we have more than the two Eids. We also have 52 Fridays to celebrate.
(Of course, not with balloons... they didn't exist in 7th century Arabia. Evil innovation!