Well I don't have any suggestions for keeping the hat on except just keep at it. But I will suggest using one of those pacifier clips to keep track of the hat. We did that when we took the kids to an amusement park. We cliped it to the back of their shirt so if the hat fell off (or was taken off) it wouldn't get lost.