I think, initially, it would only appeal to a very limited segment of the population, depending on where you live. If the restaurant ended up having really outstanding food and/or service and your reputation spread by word of mouth then you would overcome that and people would find it cool to go to a place with that name.
I think there is way more power in the name and image of a restaurant than people believe. For example, in our town there used to be a restaurant called: "The XYZ Steakhouse" (not the real name but I don't like to identify myself too much online). The food was AMAZING. It was in a century home and really fit in with the neighbourhood. Had awesome potential. But the inside was decorated like an English tearoom right down to the Royal Doulton figurines. My mother is English so this style is normal for me but in a steakhouse it was just totally wrong. It only lasted a couple of years which was a real shame because it was really great food.
So, if you want to appeal to only "crunchy" types and your town can support that, I say "Go for it". But if you want to play it safe, I would go for something a little more neutral.
p.s. Just for the record, I *would* actually choose to try out a place with a name like that.
p.p.s. Like a PP, it does conjure up images of carob, tofu and hemp seeds for me.