OP, from what you have written, it seems as if he is seeking your connection and comfort, especially after being away from you, whether he has been in another room or away at the childminder, it's a very healthy need for a child to want to be with mother, it is instictive to survival.
In western culture we tend to view breastfeeding as nutritive, but it is so much more. The non-nutritive suckling fulfills so many needs, especially for child who is away from mother for a period of time. He may have difficulty unlatching for fear of you leaving his side, but the more his need is fulfilled the more he realizes you are there to trust, love and comfort him.
I was reading in a Le Leche League book, Playful Learning, a good analogy pertaining to fulfilling a child's needs......something along the lines......... if your child is cold and needs a coat, we put one on, need fulfilled....when a child needs comfort of mother's presence it's also a need, and unless fulfilled will always be there......or similar...along those lines.
Fulfilling the child's needs satiates him instead of him yearning for more, and struggling to get that needed connection and comfort, very basic needs. Breast means mother to a child, when they want breast, they also want mother.
Long story short, I would roll with it until he is more comfortable. Don't ever believe those who say you are spoiling him, that myth only came out in the last century or so, mainly when the 'so called' child experts starting telling parents how to 'parent'. I live in the US and in 1880, 96% of American children were breastfed 2-4 years. I doubt many back then called it 'spoiling.', they were just fulfilling their child's needs, it was well accepted. Yet today, people commonly say that about breastfeeding a child, it's doesn't make sense at. Try not to let those people get to you, they must have been reading too much 'Luther Emmet Holt', ughh, some very cruel advice, He should have stuck to medical advice instead of parenting.......never the less many people believed it and worse, passed it down generations.