Sure, I am always up for a good debate! Because I drove a little car for 15 years and my DH drives one, AND I will be commuting in one soon (as I will leave the Explorer with the carseat for MIL and DS to use during the day). (We have 2 subarus and the SUV), this is still highly relevant to me, as you can see, my family finds itself on both sides of the equation (excuse the pun).
Anyway, your point is valid. By accepting the vehicle that my mom offered us when she got a new one, I did make a decision to put myself in the larger category. It would be nice if the situation on the road here were more equitible, as it is in Europe (if my memory serves me) but alas, it is not!
ETA -- A point I wish people would focus on more is just driving carefully and respecting others on the road. No matter what the size of the vehicle, if people did not drive drunk, didn't speed, and just paid attention, as well as letting people change lanes and not cutting others off, I think there would be a lot fewer needless deaths, regardless of the mass of the vehicles out there.
And what about all those people with trailers of stuff, or stuff strapped to the top of the car. I don't know how many times stuff has come flying off when I'm on the highway. I read an article how this really increased with the whole DIY thing.