he he he...love the pencil dropping comment!
There are so many awesome replies and ideas, that I don't have much to add except that it will get better. I have four children and went through the same thing with my oldest. Mine are 9,7,4,and 2 1/2. I became very upset and frustrated sooooo many times during that time--what am I saying, it still happens!!! But, my oldest has really improved. She is very easily distracted and I still have to work with her on focusing. Some days she does awesome and gets her work done super quick. Other days it takes her an hour just to finish one tiny page!!! Talk about frustrating!
I just wanted to drop a line of encouragement for you. What you are doing is so fantastic and wonderful. Home schooling, like so many other things, just keeps changing. I am slowly learning how to not get upset when we run into a rough patch. My husband always tells me "when I get frustrated to not blame myself or feel bad, that my frustration/desire to fix the problem is because I care and am trying to do the right thing."! You will find a solution to your problems and you will both learn how to work with each other. AND what works with your oldest might not be what works with the next child!! All in all, give yourself a big hug, read as much as you can about all of the different ways to teach/interact with your children, and try a little of this and a little of that. Every thing will work out in the end!
Hugs and well wishes!