I feel your pain! My dd is the same way and has been for a while. The pediatrician told me not to worry as long as she has plenty of wet dipes and is making up for the missed feedings when with me....and she is...all.night.long. Like your son she might take 4-6 ozs and sometimes nothing at all. Daycare still offers several times a day, hoping she will take it.
So yes, I am super sleepy and trying to find a bottle/nipple combo that she will take during the day so I can get a little more consistent sleep. Try a cup- my dd isn't interested but lots of babies do well taking milk from a cup. You could try a little cereal to keep the tummy fuller but don't be surprised if your little one isn't quite ready for solids. I would have dad or grandma make it extra soupy and give it when you aren't home to tide baby over.
My dd wants her bottle really warm- its the only way she will take any at all. I guess its pretty warm when it comes out of the breast so it makes sense. Make sure dad and grandma try different temps to see what he likes. He might want it cooler because its different from momma too.
And maybe he is like my daughter and just wants momma. In that case it may just be that you will have to nurse right before you leave and again as soon as you get home.