Cut-throat academia leads to 'natural selection of bad science', claims study

Getting stuff right is normally regarded as science’s central aim. But a new analysis has raised the existential spectre that universities, laboratory chiefs and academic journals are contributing to the “natural selection of bad science”.

To thrive in the cut-throat world of academia, scientists are incentivised to publish surprising findings frequently, the study suggests – despite the risk that such findings are “most likely to be wrong”.

Paul Smaldino, a cognitive scientist who led the work at the University of California, Merced, said: “As long as the incentives are in place that reward publishing novel, surprising results, often and in high-visibility journals above other, more nuanced aspects of science, shoddy practices that maximise one’s ability to do so will run rampant.”

The paper comes as psychologists and biomedical scientists are grappling with an apparent replication crisis, in which many high profile results have been shown to be unreliable. Observations that striking a power pose will make you feel bolder, smiling makes you feel happy or that placing a pair of “big brother” eyes on the wall will protect against theft have all failed to stand up to replication.

Sociology, economics, climate science and ecology are other areas likely to be vulnerable to the propagation of bad practice, according to Smaldino.

“My impression is that, to some extent, the combination of studying very complex systems with a dearth of formal mathematical theory creates good conditions for low reproducibility,” he said. “This doesn’t require anyone to actively game the system or violate any ethical standards. Competition for limited resources – in this case jobs and funding – will do all the work.

Drawing parallels with Darwin’s classic theory of evolution, Smaldino claims that various forms of bad scientific practice flourish in the academic world, much like hardy germs that thwart extermination in real life.

One scientific “germ” identified in the paper is the problem of “low statistical power”. Typically this refers to findings in human behaviour, health or psychology based on data from too small a sample of people to be able to draw any statistically sound conclusions.

Despite red flags being consistently raised on the issue in the scientific community, the latest analysis, published in the journal Royal Society Open Science, shows that sample sizes in studies have not increased during the past 50 years.

Another problem is the publication of “false positive” results, where random noise in the data appears to be a real phenomenon of interest. Since the failure to reproduce a result rarely makes a real dent in a laboratory’s prestige, the reliability of results is only “weakly selected for”, the study suggests.

Smaldino cites an experiment by the American psychologist Daryl Bem, who purported to show that undergraduates could predict the future and published the result in a prestigious journal.

Read more https://www.theguardian.com/science/2016/sep/21/cut-throat-academia-leads-to-natural-selection-of-bad-science-claims-study

Courtesy of Guardian News & Media Ltd