It depends on where you live. We live in an area that has been reported in books as having a lot of vog, but actually it does not. We've been here 2 years and there was a 2 week period where the trade winds changed and I could actually taste the sulphur in the air and I had a scratchy feeling in my throat from the vog. During that period they closed the road near the Volcanos Natonal Park because of the volcano activity and all the stuff that was in the air. We live about an hour away from the park, but our air quality was still affected. From our house, you can usually see all the way to the ocean, about 4 miles away. When the vog is hanging it is hazy and it looks foggy, all you can see is greyish blue where the sky looks like it meets the ground instead. We can usually see the ocean.

Our friends moved away because they had 3 people in their home with asthma issues and they were having too many problems. I get migraines related to the sulphur, so when the volcano was acting up and the winds were blowing the air toward us, I had headaches for those 2 weeks. I also get headaches whenever we have to go near that area of the island if there is a lot of sulphur in the air.
If you live near the volcano, you tend to have vog and air issues.
Some of the areas where housing is more expensive actually have vog issues because the trade winds blow the vog there.
The best thing you can do is visit several times a year and see for yourself, imo. The way it is for me, whenever we drive in to either Hilo or Kona, I start getting stopped up and my eyes, throat and nose start itching and getting irritated. I feel like the vog is more of an issue in other areas than it is where people
say it is. You really just have to be here a while and see it for yourself.