I have had that before and they can be very painful. I had it lanced by a nurse, then soaked in epsom salts 2-3 times a day for a few days.
Over the next few years, it did return a couple times, but didn't get as bad. It seems to form again when irritated so if I had underwear or a pad or something that felt at all uncomfortable, I'd change. This seemed to help them from becoming really awful again and they eventually just went away on their own over a few days' time.
However, the last time it came back, it got pretty painful again and I noticed that it looked like it was about ready to go. So I soaked in very hot water with epsom salts to get it to open up on its own.
Personally, if I got them often (especially if they didn't get better easily on their own), I would consider having it surgically removed.
I am a 40 year old unschooling, belly dancing, artist-mama of one almost 8 year old. I just had brain surgery and about it a bit because it's just so surreal.