I do remove the solid fat from the refridgerated broth because we don't like fat floating on the top of our soups, etc. But I save the fat and use it. It's good stuff! And I don't skim until I'm ready to use the broth, because it helps to keep the broth fresh longer.
I don't use the fat from my chicken broth, though. It always seems a bit rancid to me. But again, I leave it on the broth until I'm ready to use it.