publication and citation bias

[kathymuggle]

There is a tendency to publish and cite studies that show positive results (statistically significant) than negative (null/void) or inconclusive studies 

This is called publication bias and citation bias.

A few articles to get people started:

http://www.bmj.com/content/315/7109/640

http://bmg.cochrane.org/addressing-reporting-biases

http://blogs.trusttheevidence.net/category/blog-keywords/publication-bias

The last article mentions that publication bias a big issue in pediatrics.

From cochrane:

"Reporting biases arise when the dissemination of research findings is influenced by the nature and direction of results. Statistically significant, ‘positive’ results that indicate that an intervention works are more likely to be published, more likely to be published rapidly, more likely to be published in English, more likely to be published more than once, more likely to be published in high impact journals and, related to the last point, more likely to be cited by others. The contribution made to the totality of the evidence in systematic reviews by studies with non-significant results is as important as that from studies with statistically significant results."

Bolding mine.  I bolded it because it highlights why we should care about publication bias

[Rrrrrachel]

I was reading about this the other day, Kathy. What influence do you think this has on vaccine research as a whole? I know what I think but I'm interested in what you have to say about it.

[kathymuggle]

I don't know yet.  

I knew publication and citation bias existed - but only in a fuzzy out-there sort of way until a few days ago.

At this moment in time ( and this is not a firm or fully formed position)  I would say people have to make decisions on the studies they see.  What else can they do?

On a personal level, I am suffering from study fatigue.  Who made a study?  Is it well designed? How can we know?  Who funded it?  Any major conflicts of interest?  (all rhetorical questions)…and now, are we missing studies because of publication and citation bias? 

I am starting to think the answers are unknowable.  It is too big a ball of wax to unwind.  This leaves one with falling back on…. something.  For some that might be doctors, for others it might be stats they strongly believe to be true (say CDC), spirituality and religion, general beliefs about medicine, their own observations and probably some other stuff I am missing.

In any event, knowing it (publication and citation) bias exists is a good thing. It seems some people are working on it, although how effective solutions will be are still up in the air.

From trusttheevidence:

"Several initiatives have been spearheaded to help reduce publication bias. The creation of open-access journals have shifted the focus from the importance of the results... to the methodological rigour by which the study was completed.

But more importantly has been the creation of online trial registries, such asClinicalTrials.gov launched in 2000. ..Registration is optional, however in 2005 the ICMJE made registration of clinical trials as a pre-requisite of publication...it sent a strong message of the importance of registration."

Edited by kathymuggle - 11/27/12 at 10:05am

[Rrrrrachel]

I posted a piece a few days ago from sciencebasedmedicine about transparency in clinical trials. It addressed some of these same issues. You might like it. If you can't find te post on here it was a relatively recent post on the blog.

I hear you on the study fatigue thing. I think out decisions are definitely colored a lot by our life experiences and a healthy dose of intuition, not just the research.