We started fostering when DD was 18 months. We didn't know to try and stay with kids her age or younger. Our first kids were 6 and 9. It went ok. There were a few issues, but they were ones that I was expecting. The biggest thing I can say for this aspect of fostering is to
make sure to spend time with your child each day. Hold them a lot. These other kids can eat up a tremendous amount of your time and energy before you know it. It's very easy for your kid who is not as special needs and demanding, to get shoved to the back burner. Not only in dealing with them in your home, but in school activities, doctor visits, mental health visits (required in my state, don't know about yours) and their family visits. In my state, I had to take them to any and all family visits, doctors, court, etc. We were gone 5-6 nights a week, plus any day stuff like court. My DD is now
exceptional at dealing with office visits of whatever sort.

It is
so important to make sure your child is still getting what they need from you. Sometimes that requires putting the foster kids on the back burner for a few minutes or hours. But it's better than adding another child with issues to world. The foster kids have lots of people out there to help them. Your child has only you! Don't ever forget that!
That said, our DD did very well. She seemed to have no trouble with kids coming and going. WE were still there, still her 'security blanket'. As she got older we told her that these kids had parents who couldn't take care of them for one reason or another, so we were going to do it for a while until those parents or some others could. It was usually a little more specific, depending on each case, but that was the gist of it.
On another note, make sure your lifestyle is fully known to your SW. I don't mean every little juicy detail, but........ We homeschool. I never thought anything about it. I didn't hide it at all, but in retrospect, I never talked about with DFS either. It just never came up. Then DD hit school age, and stayed home. It turns out the local supervisor is vehemently against homeschooling. We lost the foster kids we had had for 3 years. We were 3 days away from the TPR hearing. The lady was harrassing us so badly, we finally moved out of state to protect our family and don't foster anymore. As much as I loved fostering and as much as we badly want to adopt, it's not worth losing our daughter. We always wanted a large family, and DD is apparently all we can have. So it goes.
If you decide your not comfortable with fostering, you might see about respite care. At least in our state, you could do just respite care while waiting to adopt. That gives you some exposure to these kids and what they're like on a small scale.
As long as all the bases are covered, it's the most wonderful experience! I don't regret our work with the kids at all! I miss it a lot.