Are you fooling yourself?

I am terrified of Alzheimer’s disease. I’m sure you have a story about this terrible ailment, but mine is that of my grandmother, who went from a happy, wise and joyful woman to someone who did not recognise her family. It’s my sincere hope that we cure this disease in my lifetime. So I read a new book, Doctored, by Science magazine writer Charles Piller, with some trepidation and alarm.

Piller covers the failures of academic medicine to discover the cause of Alzheimer’s disease, spending decades and millions of dollars (perhaps billions!) in futility. One of the major stories in the book is the most recognisable to me as a chemist – the story of Cassava Sciences and their leadership. They advanced their clinical candidate simufilam based on data that was clearly compromised. One of the scientists, Hoau-Yan Wang, was the originator of much of the bad data.

I have seen this kind of organisational cognitive error before. Good news from the scientists is never looked at very closely, and because of that, further clinical work and investment pitches can be based on results that are not actually true. The entire organisation has a financial incentive to ignore contrary data and focus on desirable, positive results.

I think this is also the incentive for bad results (whether in genuine error or in deliberate falsehood) to be promulgated in chemical manufacturing. Almost all commercial chemical manufacturing is attached to large amounts of money. It is certainly possible for results to be made up, or at least for small details to be elided on purpose to make a sale happen. If someone in leadership leans really hard on the quality control department or on the quality assurance department, documents could be lost or faked. A smart customer will do their own testing along the way – it will catch far more legitimate errors from vendors than deliberate falsehoods, but it is a great way to make sure that you’re standing firm in chemical truth.

Doctored has story after story about faked photographs. Molecular biology relies on the humble Western blot, or the use of protein electrophoresis to separate out proteins by size (blots often look like misshapen or particularly messy thin-layer chromatography streaks with oval-shaped dots where the proteins are detected). The photographs of Western blots that appear in papers are typically scanned and then often edited to make them clearer. This is where fakery is often detected, where lanes within blots are cut out or copied to demonstrate the presence or absence of a protein.

 The ubiquity of graphic editing software makes this kind of manipulation far too easy

Chemistry is certainly vulnerable to this kind of trickery. The most common kind that I have seen is in nuclear magnetic resonance spectra where it is clear that peaks of impurities have been removed and the baseline is unnaturally smooth. I think the ubiquity of graphic editing software makes this kind of manipulation far too easy. Requiring FIDs and other electronic files that demonstrate the original data so that the original spectra can be reproduced by other groups should be a very important part of ending this kind of fakery.

The biggest contrast I feel between chemical manufacturing and the science of Alzheimer’s disease is the relative simplicity of verifying results. Even if you are a chemist in graduate school, you should do as my company does – every raw material, no matter how simple, is checked for its identity. Very few materials are accepted on vendor certificate of analysis and most of it is tested for identity (is this really sodium hydroxide?) and satisfactory purity (how good is this 4-chloro-2-fluorobenzene?).

Recognise that you are the easiest person to fool

While it is possible to request biological samples from other laboratories and attempt to reproduce their findings, this is a lot more difficult than basic reproduction in chemical research, where you order the various compounds from the local laboratory chemistry company, follow the procedure in the supplementary information, and do the chemistry yourself.

I wonder if this basic ease tends to cut down on making up results out of whole cloth within chemistry. That said, you’re probably never going to get around to reproducing step 23 of a published 34-step total synthesis, and there the reported 74% yield is vulnerable to claims that this number is potentially inflated.

The Latin phrase ‘Caveat emptor’ or ‘Let the buyer beware’ is true for all of life and much of science. When you are reading papers or patents, recognising that you are the buyer is important and you should be aware that the fantastic results you are reading about may be false. Even more importantly, before you run that reaction or report those great results, recognise that you are the easiest person to fool and take a moment to look over your data one more time.