oh yes, sometimes it's bed time even for you too whether you like it or not. they have a secret radar that knows when you sneak away.
ever do the ultra ultra slow motion get out of bed move? where you move like a milimeter a minute, freezing ever so often and staring at your child to make sure they're still asleep, then moving slightly further...and still slightly further...ever so slowly...so gentley...slightly further some more...until... hooray! you're up! and he/she's asleep! then you take two steps of freedom towards the door... then PSYCH! "don't think so, mama!" awake again. and nothing but nothing returns them to peaceful slumber land but the warmth of your body and comfort of your breast. and you start the whole dance again, likely falling asleep yourself this time passing up your potential "time to yourself" time.
i know it's not convenient, but it is natural. and it honestly doesn't last forever. you'll never look back years down the road and regret those nights of staying there and holding and nursing them as long long long as it can take. it can be hard in the moment, but it does pass. as i sit here and write this my 4 year old and 2 1/2 year old are sleeping (though that could indeed change at any moment).
you just sometimes have to surrender to it and remind yourself of the big picture. and if it's any help to look at it this way too, this "radar" of when you leave is actually a very healthy survival skill---historically humans haven't had the "luxury" of being able to leave their babies to sleep away and alone from mommy even for a moment. babies who were left alone were very vulnerable. it's healthy for baby to wake and seek your breast and say "hey, don't leave me!".
so i don't have any magic advice, other than to empathize with you and remind/tell you that it WON'T last forever!