Cautionary tales from the front lines of science
I teach a short unit on scientific ethics in my senior design class, so when I was offered a review copy of the book, On Fact and Fraud, by David Goodstein, I took it. The title of this essay is in fact the subtitle to Professor Goodstein's book.
The book is composed largely of essays that Goodstein had published elsewhere, and the cautionary tales are mostly from physics. Among other anecdotes, Goodstein discusses Millikan's oil-drop experiment, cold fusion, the case of Jan Hendrik Schön at Bell Laboratories, a little-known case where a Caltech postdoc apparently falsified a figure in a paper, and the successful discovery of high-temperature superconductivity.
Two of these, it seems to me, are not exactly cautionary tales. The Millikan story, I thought, belonged more in a history-of-science book than in a book that claims on the dust cover to be "a user's guide to identifying, avoiding, and preventing fraud in science." Indeed, the only protagonists I would caution in the Millikan chapter are the reporters who may have tried too hard to find misconduct where none existed. And I found nothing cautionary in the tale of high-temperature superconductivity. Schön presumably faked his data, and Pons and Fleishmann, the "discoverers" of cold fusion, presumably did not; Goodstein makes much of these tales, as well as the story of the postdoc.
Early on, Goodstein presents 15 precepts for preventing scientific fraud -- then demolishes every one of them. He briefly discusses the etiology of scientific fraud and notes that most people who commit fraud probably think they are right, because if they were wrong they would almost certainly be discovered. So, as he says, they think they are reporting truths but have dispensed with all the messy experimentation that other scientists think is important. Unfortunately, he can give no prescription for preventing scientific fraud.
Goodstein observes that scientific fraud is associated with three risk factors: career pressure, but rarely what he calls "simple monetary gain"; "knowing" the answer; and working in a field where precise answers are hard to come by. I think he may have left one or more out: working in a laboratory in which the principal investigator is not involved in day-to-day supervision, and possibly working for a sponsor who has a vested interest in the outcome. Pertinently to readers of PT, he does not discuss, for example, intelligent-design creationism (nor HIV denial nor global-warming denial), which is arguably an example of scientific fraud.
Unfortunately, much of the action today, both in fraud and in science, is not in physics but in biology or medicine. It is too bad, then, that Goodstein did not include such cases as that of Woo-suk Hwang, whose papers contained fabricated data and who was additionally convicted of related crimes, or Andrew Wakefield, whose disastrous paper on the MMR vaccine had to be withdrawn. Indeed, Goodstein notes that most instances of scientific fraud involve biomedicine, possibly, he thinks, because MD's are not as well trained in research as are PhD's (but see also his third risk factor). Goodstein, perhaps wisely, perhaps not, follows the adage to write about what you know. What he knows includes Millikan, some of the protagonists involved in cold fusion, high-temperature superconductivity, and Caltech. I liked the book, but I thought that it would have been better if he had gone a little farther afield.
45 Comments
Ntrsvic · 21 May 2010
I am a third year post-doc and I have to say, that I have personally been both a 1st and 2nd hand witness to scientific fraud.
1st. In my grad school lab, we had one student completely fake her PhD. And when she was confronted, she would only show the general assay in front of people, and pulled the bait and switch on the direct NMR assays.
2nd. Our collaborates in grad school, were the ones that discovered that Hellinga's denovo enzyme design of TIM was the result of poor experimental purification (A. not using a knock-out strain of e. coli and B. not using an gradient purification of the denovo enzyme) But the really suspect thing for me about this was, If you read Helinga's now retracted paper, the kM for his denovo TIM is all most exactly that of e. coli's TIM, but surprisngly enough, e. coli TIM's (probably the most studied TIM) kcat and kM was not listed in the paper (something that was obvious to do), which indicates to me that he knew. Especially in light of the way he through the grad student who did this work under the bus when this retraction went down. Luckily for her, she kept copies of 1. her notebooks and 2. notations of her objections to the publication in the first place.
Anyways, these experiences made we want to right a book on this exact same topic, but I have neither the time nor motivation .../sad panda.
Ntrsvic · 21 May 2010
http://www.ncbi.nlm.nih.gov/pubmed/15218149
The paper with the listed retraction
*when a non-ecoli enzyme is purified from e. coli that also has that function in e. coli, having the non-ecoli enzyme's kM match the one from e. coli combined with a week kcat is a very strong indicator of e. coli enzyme contamination.
sparc · 21 May 2010
eric · 21 May 2010
What, no Blondlot? No Piltdown?
Peer review is good for catching errors, but the the ultimate scientific quality check is reproducibility. In the n-ray case, for example, what made the folks at Nature suspcious was when several teams of scientists tried umpteen times to repeat the experiment and it just never gave the expected result.
Fast, independent repetition may not prevent fraud, but it'll help reveal to the scientific community that we have a problem fairly quick. And it may have a deterrent effect on potential fraudsters to know someone's going to repeat their experiment and compare the results right away.
Joe Felsenstein · 21 May 2010
Interestingly, there are almost no cases of scientific fraud in the field(s) I work in ... because these fields (methods of phylogeny inference and theoretical population genetics) are theory. The reader can, in principle, follow the proofs line by line. I can think of one major researcher who used to drive everyone crazy by publishing theorems and saying that the proofs would be in a later paper. He never did publish most of those proofs, but when others have tried to verify the theorems, they have been able to reconstruct them, and yes, his theorems were in fact correct.
There are of course cases where people made exaggerated claims about the importance of their results, and of course lots of cases where they used incorrect methods and got wrong results. But in the latter cases they honestly believed in their results.
eric · 21 May 2010
Henry J · 21 May 2010
Then there's pier review, which helps keep people from missing the boat...
GvlGeologist, FCD · 21 May 2010
Olorin · 21 May 2010
Robert Park's book, "Voodoo Science" is still relevant, although some of his examples are dated.
Of particular interest is his depiction of how the "discover" of a new effect comes to believe in his result so strongly that his belief crosses the line into palpable delusion. Blondlot's N-rays. Joe Newman's perpetual-motion motor. Need I say Pons & Fleischmann?
MrG · 21 May 2010
Matt Young · 21 May 2010
Matt Young · 21 May 2010
harold · 21 May 2010
jswise · 21 May 2010
Another good book is Walter Gratzer's The Undergrowth of Science: Delusion, Self-Deception, and Human Frailty. I guess these books all tend to cover the same incidents, though.
John Vanko · 21 May 2010
Yet another good book: "Bad Science: The Short Life and Weird Times of Cold Fusion" by Gary Taubes, 1983, convinced me that Pons and Fleischmann did indeed commit fraud.
John Vanko · 21 May 2010
Oops! 1993
Richard · 21 May 2010
Another good book on this issue, albeit now dated, is Betrayers of the Truth: Fraud and Deceit in the Halls of Science by William J Board, 1983.
Ichthyic · 21 May 2010
Alex H · 22 May 2010
Anyone else get offended by the way that the deniers are always labeled "skeptics" despite the fact that their position is so far away from true skepticism that there's an observable red-shift?
Ichthyic · 22 May 2010
Anyone else get offended by the
way that thedeniersyes.
their very existence offends me.
:)
Bob O'H · 22 May 2010
Ron Okimoto · 22 May 2010
Can intelligent design be considered to be scientific fraud? The only "scientific" example where they published the propaganda switch scam junk was in Meyer's systematics paper that was disowned by the journal that it was published in. It seems that all that got published in that paper was the smoke screen obfuscation junk. I don't think that any of the intelligent design scam junk was ever published in a science journal. Intelligent design turned out to be fraud, but it wasn't scientific fraud because they never published the fradulent material in science journals as legitimate science. They just scammed the ignorant public with it. Beats me if they ever fooled enough scientists to matter. It turned out that the ID perps knew that the intelligent design junk never made the grade. Nelson was the first to admit that they never had a scientific theory of intelligent design. Even Philip Johnson admitted that intelligent design had never made the grade and pointed at the "science" ID perps and claimed that they hadn't come up with any ID science worth teaching.
So the ID perps knew that they came up short. They ended up running the bait and switch scam on their own creationist support base rather than put up their ID science to teach in the public schools. The switch scam doesn't even mention that ID ever existed. Every rube school board and legislator that bought into the ID scam and that has claimed to want to teach the science of ID has had the bait and switch run on them. This is bogus fraud, but not the type written about in the book. ID was more of a dishonest political scam than an attempt to fool scientists.
Gary Hurd · 22 May 2010
Something I have noticed over the years is that it is hard to distinguish in some cases between fraud and actual incompetence.
Jesse · 22 May 2010
My understanding of the whole cold fusion fiasco is that it started off as incompetence and then turned into fraud. The third guy who worked on that (the one who Pons and Fleischmann published without) is also a truther. He even went so far as to make the argument that the 2nd Law of Thermo proves that the towers could not have fallen as they did without human intervention.
eric · 22 May 2010
Matt Young · 22 May 2010
Dave Luckett · 22 May 2010
There are things like the Ica Stones, almost certainly originally manufactured as curios, without perhaps an intent to actually commit fraud, and indeed the original creators don't seem to have made anything much out of it. The question is, who spun the ridiculous fancy that they were Precolumbian and demonstrated that the Incas lived with dinosaurs?
There's that hoary creationist chestnut, "Piltdown Man". That was a fraud, certainly, and probably originally intended to be one, but the question is whose it was, and on whom?
There used to be a cottage industry in ports in Europe, creating what were called "Jenny Hanipers" - artfully sewn and stuffed skins and different animal parts, made to look like one animal. "Mermaids" and "batfish" were favourites, as I recall. The gentlemen of the Royal Society quite reasonably thought that someone was making game of them, the first time they saw a platypus, mounted and stuffed, after its seven-months journey back to Britain.
Jesse · 22 May 2010
Jesse · 22 May 2010
But I should add that the refusal to retract after it was shown that they were wrong is where it became fraud. Jones lucked out that he wasn't included.
Dave Luckett · 22 May 2010
Wishful thinking can sometimes overcome even a strong committment to fact. I remember thinking when the story broke, and everything looked kosher - "They've done it! Stars, here we come!"
Ayyy. It was too good to be true.
Moses · 22 May 2010
John Vanko · 22 May 2010
Steve Jone of BYU initially published on muon-catalyzed cold fusion, a reproducible and perfectly legitimate subject in physics explained by present-day, main-stream theory. In his Scientific American article he clearly states that the energy released is insufficient to create a new muon and thus sustain a chain reaction. No new energy source here.
In later years he may have gotten caught up in the general cold fusion debacle, but in his early days he was entirely legitimate.
Moses · 22 May 2010
Jesse · 22 May 2010
Ron Okimoto · 22 May 2010
Frank J · 23 May 2010
Matt Young · 24 May 2010
MrG · 24 May 2010
Malchus · 24 May 2010
MrG · 24 May 2010
eric · 24 May 2010
MrG · 25 May 2010
harold · 25 May 2010
Henry J · 25 May 2010
Ron Okimoto · 30 May 2010