I guess it all depends on how you define atheism.
I am a strong atheist. I don't believe that "there is nothing." I believe that everything is natural, rather than supernatural.
The literal definition of an atheist is someone who believes that gods don't exist. That's me.
One can be an atheist and also be agnostic. An agnostic acknowledges that it's impossible to know, either way. Knowledge and belief are two different things. If you knew, you wouldn't need to believe. Truth doesn't require faith.