Advertisements

When science gets it wrong: Let the light shine in; Two big recent scientific results are looking shaky-and it is open peer review on the internet that has been doing the shaking

When science gets it wrong: Let the light shine in; Two big recent scientific results are looking shaky—and it is open peer review on the internet that has been doing the shaking

image001-4

Jun 14th 2014 | From the print edition

SCIENTISTS make much of the fact that their work is scrutinised anonymously by some of their peers before it is published. This “peer review” is supposed to spot mistakes and thus keep the whole process honest. The peers in question, though, are necessarily few in number, are busy with their own work, are expected to act unpaid—and are often the rivals of those whose work they are scrutinising. And so, by a mixture of deliberation and technological pressure, the system is starting to change. The internet means anyone can appoint himself a peer and criticise work that has entered the public domain. And two recent incidents have shown how valuable this can be.

The first concerns pluripotent stem cells, the predecessors of every other body cell. Pluripotent cells interest doctors and biologists, who hope to use them to investigate diseases, test drugs and, eventually, regrow patients’ damaged body parts.

At first, such cells had to be extracted from embryos, an ethically dodgy process if those embryos were human. Then a way of making them from skin cells was invented. This, though, involves fiddling about with proteins called transcription factors, and is finicky. So when, in January,Nature published a pair of papers describing a simpler way of making them, people around the world sat up. In these papers Haruko Obokata of the RIKEN Centre for Developmental Biology in Kobe, Japan, and her team, claimed to be able to create pluripotent cells by exposing ordinary, non-stem cells to weak acids, physical squeezing and some bacterial toxins.

No sooner had the papers been published than doubts emerged. Others tried to replicate Dr Obokata’s methods and failed—and made their findings known on the web. Other blogs and websites published apparent irregularities in diagrams and pictures in the papers, and pointed out that one passage seemed to have been copied verbatim from elsewhere.

In April a RIKEN committee concluded that Dr Obokata had twice manipulated her data in an intentionally misleading fashion, something they classed as research misconduct. The more light that was shone on the papers, the weaker the claims became. Although Dr Obokata had defended her work, her co-authors were divided on whether formally to retract the papers. On June 4th she reportedly agreed to withdraw both (although as The Economistwent to press both were still available, unchanged, on Nature’s website).

Shrinking BICEPs

The second claim came from cosmology. On March 17th researchers from the Harvard-Smithsonian Centre for Astrophysics, led by John Kovac, held a press conference at which they announced that they had discovered interesting patterns in the cosmic microwave background, a type of weak radiation left over from the universe’s earliest moments. They said they had spotted the signatures of primordial gravitational waves, ripples in space formed just after the Big Bang.

Once again, it was big news (including in The Economist). The existence of such waves would give strong support for the theory of inflation, which holds that the early universe underwent a brief burst of faster-than-light expansion. Inflation was put forward in the 1980s by theorists as a way to resolve various knotty problems with the standard theory of the Big Bang. But although it is widely assumed to be true, direct evidence that it happened had been lacking.

Dr Kovac and his colleagues made much of their data available online at the time, prompting hundreds of physicists to check their work. Doubts soon surfaced. The team’s claim to have spotted the waves relies on them having diligently scrubbed out every possible source of false positives. But doing that is hard, because the most likely culprit—interstellar dust—is poorly understood. Such diligence is made doubly difficult by the fact that, although several teams are hunting for primordial gravity waves, the glory of being the first to spot them means none is willing to share its data with the others.

The various online arguments culminated with the publication of an online paper by researchers from New York University, Princeton University and the Institute for Advanced Study (also in Princeton). This concluded that Dr Kovac’s data, which came from an Antarctic telescope called BICEP-2, may well have been contaminated by space dust, and that the purported gravitational waves may be much weaker than the team first claimed—if they exist at all.

No one is accusing Dr Kovac and his colleagues of the sort of sharp practice for which Dr Obokata was censured. Instead, the argument is about whether their results were really solid enough to justify the rapturous reception they were given, says Katherine Mack, an astrophysicist at the University of Melbourne. But both cases reflect the rise of open, post-publication review on Facebook and Twitter, by e-mail, on blogs, and in the comments sections of websites like arXiv, which hosts preprints of papers in physics and mathematics. As Paul Knoepfler, a biologist at the University of California, Davis, whose blog was used to co-ordinate the efforts of those trying to replicate Dr Obokata’s work, puts it, “I suspect that if published even five years ago, the [stem cell] papers’ serious problems would have gone unnoticed for far longer.”

The BICEP-2 results are an extreme example of this trend. They were announced without having first been published in a journal. This means that all the reviewing took place after the fact. Doing things this way makes some scientists nervous, especially when the subsequent discussion throws up problems. Blogs, comment threads and Facebook discussion groups mean that some of the debates which used to happen behind closed doors now take place in public, where anyone can watch.

The public, however, pay for most of this stuff. That open peer review gives them a glimpse into the reality of life inside the ivory tower is probably a good thing. Despite the activities of people like Dr Obokata, science is one of the most trustworthy human activities. But as Ronald Reagan put it in a different context, “Trust, but verify.”

 

Advertisements

About bambooinnovator
Kee Koon Boon (“KB”) is the co-founder and director of HERO Investment Management which provides specialized fund management and investment advisory services to the ARCHEA Asia HERO Innovators Fund (www.heroinnovator.com), the only Asian SMID-cap tech-focused fund in the industry. KB is an internationally featured investor rooted in the principles of value investing for over a decade as a fund manager and analyst in the Asian capital markets who started his career at a boutique hedge fund in Singapore where he was with the firm since 2002 and was also part of the core investment committee in significantly outperforming the index in the 10-year-plus-old flagship Asian fund. He was also the portfolio manager for Asia-Pacific equities at Korea’s largest mutual fund company. Prior to setting up the H.E.R.O. Innovators Fund, KB was the Chief Investment Officer & CEO of a Singapore Registered Fund Management Company (RFMC) where he is responsible for listed Asian equity investments. KB had taught accounting at the Singapore Management University (SMU) as a faculty member and also pioneered the 15-week course on Accounting Fraud in Asia as an official module at SMU. KB remains grateful and honored to be invited by Singapore’s financial regulator Monetary Authority of Singapore (MAS) to present to their top management team about implementing a world’s first fact-based forward-looking fraud detection framework to bring about benefits for the capital markets in Singapore and for the public and investment community. KB also served the community in sharing his insights in writing articles about value investing and corporate governance in the media that include Business Times, Straits Times, Jakarta Post, Manual of Ideas, Investopedia, TedXWallStreet. He had also presented in top investment, banking and finance conferences in America, Italy, Sydney, Cape Town, HK, China. He has trained CEOs, entrepreneurs, CFOs, management executives in business strategy & business model innovation in Singapore, HK and China.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s

%d bloggers like this: