Thrush, like yeast, is a fungal infection and they are treated similarly. It does usually spread to the mouth (and mom's nipples) but not always.
The "least medicinal" way to treat it is with a vinegar bath (it doesn't hurt, don't worry!- pour a little vinegar into the bath water after dc has been washed and let sit for a little while, then rinse) and then lots of diaper free time. Fresh air and light kills yeast. At least a half-hour, but as long as you can go. A little baby pee isn't bad to clean up. Sit dc on some towels and let the air "get at it" while you play or read stories.
Olive oil would not be good for it. In fact, you should use the very minimum in the diaper area as possible. If you are using prepared wipes of any kind, stop. They contribute to rashes and fungus, as they trap moisture (as does olive oil, petroleum jelly, and the likes) and made-at-home solutions as well as store bought ones use unneeded chemicals and lotions. All you need is water for a damp washcloth to wipe up and a dry one to pat the bum after a clean. Be sure to throuroughly wash your hands after each diaper change. Less is best.
Also, if you are using cloth, you should strip them and wash them with vinegar as well and hang them out to dry in the light. Just a run through the washer won't kill yeast. No matter what kind of diaper you use, change them frequently.
Finally, if all this doesn't work, you can use Lotrimin or see your doctor for nystatin.