Advertisements

When science gets it wrong: Let the light shine in; Two big recent scientific results are looking shaky-and it is open peer review on the internet that has been doing the shaking

When science gets it wrong: Let the light shine in; Two big recent scientific results are looking shaky—and it is open peer review on the internet that has been doing the shaking

image001-4

Jun 14th 2014 | From the print edition

SCIENTISTS make much of the fact that their work is scrutinised anonymously by some of their peers before it is published. This “peer review” is supposed to spot mistakes and thus keep the whole process honest. The peers in question, though, are necessarily few in number, are busy with their own work, are expected to act unpaid—and are often the rivals of those whose work they are scrutinising. And so, by a mixture of deliberation and technological pressure, the system is starting to change. The internet means anyone can appoint himself a peer and criticise work that has entered the public domain. And two recent incidents have shown how valuable this can be.

The first concerns pluripotent stem cells, the predecessors of every other body cell. Pluripotent cells interest doctors and biologists, who hope to use them to investigate diseases, test drugs and, eventually, regrow patients’ damaged body parts.

At first, such cells had to be extracted from embryos, an ethically dodgy process if those embryos were human. Then a way of making them from skin cells was invented. This, though, involves fiddling about with proteins called transcription factors, and is finicky. So when, in January,Nature published a pair of papers describing a simpler way of making them, people around the world sat up. In these papers Haruko Obokata of the RIKEN Centre for Developmental Biology in Kobe, Japan, and her team, claimed to be able to create pluripotent cells by exposing ordinary, non-stem cells to weak acids, physical squeezing and some bacterial toxins.

No sooner had the papers been published than doubts emerged. Others tried to replicate Dr Obokata’s methods and failed—and made their findings known on the web. Other blogs and websites published apparent irregularities in diagrams and pictures in the papers, and pointed out that one passage seemed to have been copied verbatim from elsewhere.

In April a RIKEN committee concluded that Dr Obokata had twice manipulated her data in an intentionally misleading fashion, something they classed as research misconduct. The more light that was shone on the papers, the weaker the claims became. Although Dr Obokata had defended her work, her co-authors were divided on whether formally to retract the papers. On June 4th she reportedly agreed to withdraw both (although as The Economistwent to press both were still available, unchanged, on Nature’s website).

Shrinking BICEPs

The second claim came from cosmology. On March 17th researchers from the Harvard-Smithsonian Centre for Astrophysics, led by John Kovac, held a press conference at which they announced that they had discovered interesting patterns in the cosmic microwave background, a type of weak radiation left over from the universe’s earliest moments. They said they had spotted the signatures of primordial gravitational waves, ripples in space formed just after the Big Bang.

Once again, it was big news (including in The Economist). The existence of such waves would give strong support for the theory of inflation, which holds that the early universe underwent a brief burst of faster-than-light expansion. Inflation was put forward in the 1980s by theorists as a way to resolve various knotty problems with the standard theory of the Big Bang. But although it is widely assumed to be true, direct evidence that it happened had been lacking.

Dr Kovac and his colleagues made much of their data available online at the time, prompting hundreds of physicists to check their work. Doubts soon surfaced. The team’s claim to have spotted the waves relies on them having diligently scrubbed out every possible source of false positives. But doing that is hard, because the most likely culprit—interstellar dust—is poorly understood. Such diligence is made doubly difficult by the fact that, although several teams are hunting for primordial gravity waves, the glory of being the first to spot them means none is willing to share its data with the others.

The various online arguments culminated with the publication of an online paper by researchers from New York University, Princeton University and the Institute for Advanced Study (also in Princeton). This concluded that Dr Kovac’s data, which came from an Antarctic telescope called BICEP-2, may well have been contaminated by space dust, and that the purported gravitational waves may be much weaker than the team first claimed—if they exist at all.

No one is accusing Dr Kovac and his colleagues of the sort of sharp practice for which Dr Obokata was censured. Instead, the argument is about whether their results were really solid enough to justify the rapturous reception they were given, says Katherine Mack, an astrophysicist at the University of Melbourne. But both cases reflect the rise of open, post-publication review on Facebook and Twitter, by e-mail, on blogs, and in the comments sections of websites like arXiv, which hosts preprints of papers in physics and mathematics. As Paul Knoepfler, a biologist at the University of California, Davis, whose blog was used to co-ordinate the efforts of those trying to replicate Dr Obokata’s work, puts it, “I suspect that if published even five years ago, the [stem cell] papers’ serious problems would have gone unnoticed for far longer.”

The BICEP-2 results are an extreme example of this trend. They were announced without having first been published in a journal. This means that all the reviewing took place after the fact. Doing things this way makes some scientists nervous, especially when the subsequent discussion throws up problems. Blogs, comment threads and Facebook discussion groups mean that some of the debates which used to happen behind closed doors now take place in public, where anyone can watch.

The public, however, pay for most of this stuff. That open peer review gives them a glimpse into the reality of life inside the ivory tower is probably a good thing. Despite the activities of people like Dr Obokata, science is one of the most trustworthy human activities. But as Ronald Reagan put it in a different context, “Trust, but verify.”

 

Advertisements

About bambooinnovator
KB Kee is the Managing Editor of the Moat Report Asia (www.moatreport.com), a research service focused exclusively on highlighting undervalued wide-moat businesses in Asia; subscribers from North America, Europe, the Oceania and Asia include professional value investors with over $20 billion in asset under management in equities, some of the world’s biggest secretive global hedge fund giants, and savvy private individual investors who are lifelong learners in the art of value investing. KB has been rooted in the principles of value investing for over a decade as an analyst in Asian capital markets. He was head of research and fund manager at a Singapore-based value investment firm. As a member of the investment committee, he helped the firm’s Asia-focused equity funds significantly outperform the benchmark index. He was previously the portfolio manager for Asia-Pacific equities at Korea’s largest mutual fund company. KB has trained CEOs, entrepreneurs, CFOs, management executives in business strategy, value investing, macroeconomic and industry trends, and detecting accounting frauds in Singapore, HK and China. KB was a faculty (accounting) at SMU teaching accounting courses. KB is currently the Chief Investment Officer at an ASX-listed investment holdings company since September 2015, helping to manage the listed Asian equities investments in the Hidden Champions Fund. Disclaimer: This article is for discussion purposes only and does not constitute an offer, recommendation or solicitation to buy or sell any investments, securities, futures or options. All articles in the website reflect the personal opinions of the writer.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: