I think limits are appropriate and necessary. Nursing is your child's first (and at this point, most important) relationship. A relationship isn't about one person, it's about two. By setting limits to allow you to enjoy nursing, you're teaching your son to respect other people, to distinguish between wants and needs, to compromise, eventually to respond empathetically to other's needs. Of course, too many or too restrictive limits can lead to an earlier weaning, but so can no limits . . . when Mommy can't stand it any more!
Â
With my twins, the first limit we set was "no nursing while Mommy is eating." That was at about 1 year. There were others as it became appropriate. We night-weaned at 2 1/2 (big limit!). I wouldn't nurse them at preschool when I was working there. At some point, I stopped nursing them simultaneously in public (try to be discreet with two very active preschoolers who are attempting to nurse while wrestling with their sibling. . . heck, forget discreet, try to keep even a tiny bit of skin covered ;-) For both my DS (one of the twins) and my current 2-year old nursling, a huge limit is "no grabbing Mommy's other nipple" - and I'm very firm about that one, to the point of ending nursing sessions because it makes me crawl out of my skin. I know there were probably other limits, too! But as they got older, we didn't need as many limits because the issues resolved themselves and they learned to respect my body, to wait until they could nurse, to accept and even prefer other forms of comfort. They nursed for 5 1/2 (DS) and 6 (DD) years, so clearly our limits didn't lead to early weaning!
Â
One thing I did try, with nursing and as much as possible in general, is to respond with "yes" instead of "no". So I might say, "Yes, you can nurse as soon as I go to the bathroom," instead of "No, Mommy has to go to the bathroom now."  I think knowing that even if they couldn't nurse immediately, they could nurse soon, really helped with accepting limits. And I always tried to evaluate the need - is the child about to have a meltdown? Then my tea can wait, we'll sit down and nurse!
Â
You mention food allergies. Do you think your DS might also have food allergies? Some of what you mention (nursing 20+ times a day), not being very interested in table foods, even being "high needs" can be signs of food allergies. MIght help!
Â
I definitely think that they sense our stress, and for many toddlers nursing is their biggest comfort. It sounds like the worst time of day is evenings when your DH is home, and you also mention getting frustrated when you leave DS w/ DH. Do you think your son might be sensing tension or conflict between you and DH? My kids all have more temper tantrums and sibling squabbles when DH and I are upset with each other, even if nothing is said out loud. If part of the issue is that DH isn't getting your frustration with the frequent nursing and your need for him to step up and be the primary parent in the evenings, maybe the two of you could sit down when DS isn't around and really discuss it, come up with a plan for DH to help in the evenings. Having a united front and a plan can also be so helpful.
Â
Good luck. Remember that no matter how long you nurse, your son has already benefited so much. He's a lucky boy.