An analysis of studies published in prominent psychological journals shows that half of the results cannot be replicated, says an article in Newsweek.

The study, released Thursday, looked at findings from 100 previously published studies in three psychological journals, Psychological Science, the Journal of Personality and Social Psychology and the Journal of Experimental Psychology: Learning, Memory, and Cognition.

The scope of the project was huge, involving 270 researchers on five continents, and was undertaken by a not-for-profit technology company, the Center for Open Science.  The company’s goal is to increase transparency and reproducibility in research.

The coordinator of the study, Brian Nosek of the University of Virginia, said it was difficult to say why the studies appear to be irreproducible.  Either team could have simply been wrong, or there may have been some methodological difference that was not originally recognized.

The researchers also say the lack of replication could be caused by cultural differences.  Participants in one country may not yield the same results as those in another, and the difference could be influenced by the time frame of the studies.

Nosek added that researchers often feel they need to make a powerful statement, which in turn can foster their own careers.  ‘What’s best for researchers isn’t necessarily what’s best for science.  The incentives are pushing us toward clean, beautiful stories. That is a wonderful thing when it’s achieved”

This was not news to the journals.  Psychological Science last year started to offer study materials for other researchers to review.  One of the problems with peer review is that it often happens too late, after the study is published.  A review of the process is underway to try to move the peer review process up the timeline.

All this is confusing to the general public, who see a study stating one position and a later review stating the opposite.  Sometimes circumstances change and that means new data has become available to improve the research, so that isn’t necessarily a bad thing.