How do you analyze evidence?

[amnesiac]

Just kinda surfing through this forum there are links to all sorts of different websites, recommendations for various books & of course, individual interpretations of scientific publications. I'm talking both pro and anti vax stuff here.

So how do you go about deciding whether or not a source is presenting the information in an accurate way? What criteria do you personally use to evaluate information you're reading? How do you go about deciding what's worthy of repeating to others or applying to your own life so you aren't classified as 'just another anti-vax nutjob' or 'just another ignorant, pro-vax bully' ?

[ERSsmom]

What a great question. I have wondered this myself before. Thank you for asking.

I sometimes have a hard time deciding what is a good resource and what is not. There is money to be made from both sides and I find it hard to trust either side. Some of the websites clearly have an agenda and that deters me from trusting them.

When I first began my research, I looked online, but it was overwhelming and I couldn't find many websites that I trusted. I do like the National Vaccine Information Center website. I trust that one more than others because their mission is not anti-vax, but pro-informed consent. So in short, the mission of the website plays a huge factor for me. Another criteria is the articles they use to back up their statements. I usually look at the actual articles (if I can). I look at whether it is a peer reviewed article or someone's opinion. I look at the sample size, who wrote it and other factors that may make the study unreliable.

Much of what I read, I won't repeat it to others as fact, I will just direct them to the site. My reasoning is to let them decide for themselves. Just because I decided something was reliable does not make it so.

As far as books go, I have found few books that I truly like on the subject. The one I always recommend is Romm's book. I know from a previous thread that you do not like Romm's books, but I found her book to be very unbiased. Even after reading it twice, I cannot decide if she is anti-vax or pro-vax. Her documentation is adequate, which is paramount. I want to be able to go to her sources and check what she is saying.

A book I read recently kept claiming that if you get Whooping Cough once you will have life-long immunity. I thought this was wrong so I googled it. I found several state health dept sites that stated that getting WC does not give you life-long immunity, although it may give you immunity for a few years. Because of this discrepancy, I was skeptical of everything else in the book. The author should research everything she includes in her book.

I'm interested in how you decide what a reliable source is.

[Gitti]

ERSsmom, that was very good and I agree with most of what you've said, especially this:

Much of what I read, I won't repeat it to others as fact, I will just direct them to the site. My reasoning is to let them decide for themselves. Just because I decided something was reliable does not make it so.

[ERSsmom]

[aira]

Not much time...

But a couple of major points for me:

1A. I trust my logic. If a "source" is emloying tactics to persuade me to their side, I generally feel that they must not have facts on their side. To be fair though, I know that some people with truthful messages also do this because it works on many people. But over time, it becomes clear if someone (or a general POV) has merit or not. If a book, paper, or site avoids actually dealing with facts or tries to make people feel silly for disagreeing with their obviously superior opinion, that's bunk.

1B. Kinda in line with the above, I do my own math, as does my DH and a couple of smart relatives. If someones stats are way off from our numbers, they become suspect. I never take a statistic at face value, b/c it is easier to manipulate them than to leave them impartial.

2. If I can find the $$$$ trail, that says everything to me. I think that anyone who spends tremendous effort to persuade is either a real crusader, or has a lot of money at stake. If there are financial ties to their own opinion, I give that opinion NO weight.

3. I do not have "trusted" sources. Well, maybe I develop my own trust in someone over a long time, but I don't listen to anyone else's "trust". It's a huge advertizing ploy. Just look at the TV commercials and note how many times the model says something like: And its from Lysol, so I trust it. It subtly gets in your brain to create mistrust in others. CDC, WHO, etc... do this non-stop.

Baby calls... gotta go!

[Gitti]

Logci!

If a book, paper, or site avoids actually dealing with facts or tries to make people feel silly for disagreeing with their obviously superior opinion, that's bunk.

I never take a statistic at face value, b/c it is easier to manipulate them than to leave them impartial.

If ..anyone who spends tremendous effort to persuade is either a real crusader, or has a lot of money at stake. If there are financial ties to their opinion, I give that opinion NO weight.

It's a huge advertizing ploy. Just look at the TV commercials ..the model says something like: And its from Lysol, so I trust it. It subtly gets in your brain to create mistrust in others.

CDC, WHO, etc... do this non-stop.

[amnesiac]

I'm interested in how you decide what a reliable source is.

If I write it myself I know it's reliable. :LOL

Seriously though, I don't use those sites like thinktwice or whale or quackbusters or any of that & I don't refer people to those sites either. I like primary research sources & textbooks. And actually I find a lot of interesting things at the CDC, FDA & EPA sites. I do find it challenging sometimes though because some things are difficult to find unless you already know what you're looking for. And if I'm reading a paper about something that's beyond my knowledge base I ask people that are actually in that field to help clarify.

Also, I'm a snob about healthcare authors. I don't particularly like it when practitioners write about things that are beyond their scope of practice - like say a chiropractor that writes about baby food or medication.

I know from a previous thread that you do not like Romm's books, but I found her book to be very unbiased.

I've only ever seen the other one that's not a vax book so I couldn't say either way. I'm not big into books like that though. I do use some reference books if I'm looking up clinical information or specific herbs or something but I never sit down & read those other types of books cover to cover.

If a book, paper, or site avoids actually dealing with facts or tries to make people feel silly for disagreeing with their obviously superior opinion, that's bunk.

And actually that's something we see here on this forum a lot - people that actually do choose to use some or all vaccines are told that they clearly have not researched the issue because if they had then they'd be nonvaxers too. So that's a big part of why I'm wondering what exactly it is we're calling "research."


What do you all think about these lists?
http://www.lib.purdue.edu/itd/techman/eval.html
http://www.chssc.salford.ac.uk/healt...h/evaluati.htm

[ERSsmom]

Originally Posted by amnesiac

What do you all think about these lists?
http://www.lib.purdue.edu/itd/techman/eval.html
http://www.chssc.salford.ac.uk/healt...h/evaluati.htm

Those lists seem good to me. I just skimmed them, sorry. But they seem like all the stuff I learned in graduate school about reliable sources. I try to apply those points when I am doing research for vax. It doesn't always work, but I do my best.

Originally Posted by amnesiac

And actually I find a lot of interesting things at the CDC, FDA & EPA sites. I do find it challenging sometimes though because some things are difficult to find unless you already know what you're looking for.

Some of the best information I have gotten from this board have been from these very sites.

Basically, I take everything anyone posts with a grain of salt. I know this board is just a bunch of mommas and grandmommas doing research for their children. I always check out the information that's given and make a decision for myself. Sometimes I like all the info given and sometimes I don't. I take what fits with me and leave the rest. I think the people on this board are a wealth of information. Just because I don't agree with everything everyone says doesn't mean I can't learn from them.

[aira]

Gitti...

OK, quoting myself here (sorry):

If a book, paper, or site avoids actually dealing with facts or tries to make people feel silly for disagreeing with their obviously superior opinion, that's bunk.

In the context I meant, people who do this without any facts on their side (real facts, not made-up ones) are promoting bunk. Honest people who understand things very well do this too (myself included, if I do say so), but I attribute that to ego, and not what I'm talking about here.

For my point, it's all in the context of their arguement.

[amnesiac]

Yes & that's really what I'm asking about - how do you go about deciding whether their facts are real or made up or misinterpreted or incomplete or whatever?

[aira]

Well I took a stab above at how I tackle whether things are made up or misused, but it is a huge task.

About whether something is complete or not, that is a real issue considering that we can't know what we don't know. My best stab at that is endless reading and asking. Maybe someone somewhere thought of some reason why what I'm seeing isn't the whole picture. Or will ask the right questions to send me in the right direcction to look for more.

At a minimum, someone might just show me that I don't know the whole picture, even if they don't shining a light on it for me.

Much of my info comes from the raw data my good buddy geneticist shares with me. I am privaledged to see stuff before any spinning happens. And her opinion is worth a lot to me, even when we disagree. She asks great questions and is a top notch researcher, with an excellent critical eye. I learn a lot from her.

I think there is no answer for how we know when something is imcomplete. I think you can't know, but you can get a sense. And it's cumulative, so sometimes with a solid knowedge base, you can spot something fishy without much trouble.

[Jen123]

I generally don't trust the sites that are full of 'personal stories'. If it were one mama to another , I'd believe it , but online , it looks like a bunch of hooey and doesn't hold water against the more agressive pro vax sites.

Anti-vax sites I'm more likely to trust if they have some of the 'bigger' names attached to them. The science , altho sometimes over my head , is correct.
They use proper terms , have numerous links (generally to the cdc , who and iom) to support their statements.

I find that some sites , pro and anti , use bully techniques. If they say 'this could happen to you' or use scare tactics (pro or anti) to dissuade your opinion...instead of letting you form your own....then those sites set off warning bells. I've seen on both pro and anti sites the banner "You child could DIE if......[fill in the blank] "

[CrazyRED]

Originally Posted by amnesiac

I like primary research sources & textbooks

Pay close attention to textbooks. The information they contain often depends on the publisher and who owns it.

[amnesiac]

Pay close attention to textbooks. The information they contain often depends on the publisher and who owns it.

What would be an example of what you're referring to?

I personally use medical, nursing, child development and microbiology textbooks to look up things like the pathophysiology of a particular condition, or common interventions for specific conditions, characteristics of specific organisms, lab values, color images, that sort of thing.

[Rainbow Brite]

I do not think there is any credible source.

[amnesiac]

So what do you base your decisions on?

(love your sig BTW!)

[Gitti]

Why you can't trust medical journals any longer

With financial ties to nearly two dozen drug and biotech companies, Dr. Charles B. Nemeroff may hold some sort of record among academic clinicians for the most conflicts of interest.

A psychiatrist, a prominent researcher, and chairman of the department of psychiatry and behavioral science at Emory University in Atlanta, Nemeroff receives funding for his academic research from Eli Lilly, AstraZeneca, Pfizer, Wyeth-Ayerst--indeed from virtually every pharmaceutical house that manufactures a drug to treat mental illness.

He also serves as a consultant to drug and biotech companies, owns their stocks, and is a member of several speakers' bureaus, delivering talks--for a fee--to other physicians on behalf of the companies' products.

From all *I* have "researched", this case is NO exception! It's more like the NORM.

Although I don't know whether it's from a reliable source.

:LOL

[mamakay]

I can't quite explain it. My BS radar just goes off sometimes, and I know the author has an agenda.
It's an intuition thing. Yeah, I know, not real scientific.
It's the truth, though.
It happens with both the pro and anti vax sources equally.

[amnesiac]

Exactly, you should never just put blind trust in any source. Like we've said before, it IS important to consider the author and the funding involved.

So Gitti, what journal articles have you researched where this has been the case & what criteria do you personally use?

[ERSsmom]

Originally Posted by Gitti

From all *I* have "researched", this case is NO exception! It's more like the NORM.

Although I don't know whether it's from a reliable source.

:LOL

:LOL :LOL