The book I have on essential oils actually recommends dabbing lavender "neat" onto blemishes. A drop on a cotton swab or ball dabbed on the blemish should be safe. The bottle of lavender I have (Aura Cacia brand) has a face spritzing "recipe" on the label. X number of drops into Y amount of water (The bottle is at home...I'm at work). If you have sensitive skin, you may need to dilute the lavender this way. But I think most people consider lavender (and TTO, for that matter) safe for using directly on the skin.
I don't have the book with me, but the author cautions against using lemon eo on the skin without diluting it...and I *think* she feels the same about rosemary eo.
HTH - If I've given you poor advice, hopefully someone will speak up and set us straight!