It's amazing how many scientific concepts I have picked up from SF. I'm with you there.
But I started with a rigorous formal grounding in the basic branches of science, did a subset of the Great Books (no normal person can do them all!) at the secondy and postsecondary levels, and that's the absolute min-bar for my kids, who are privileged to have access to prep school and two well-educated parents. (If my DH were in this thread, he would add a min-bar for formal mathematical study, and it would be waaaay above Algebra II where I quit.)