prev next

Keith Naughton


Fake Research: A Bigger Problem Than You Think

by Keith Naughton
 

Over the past few weeks a furor has gripped the world of political science. A landmark study that claimed individual voters would be more accepting of gay marriage if they had personal interaction with proponents has turned out to be a pack of lies. The study by Michael LaCour, a doctoral student, and Dr. Donald Green, a prominent political scientist, has been retracted and Dr. Green has issues a lengthy apology. http://www.newyorker.com/science/maria-konnikova/how-a-gay-marriage-study-went-wrong

The entire sordid mess is not surprising to anyone with experience in political science research and the groupthink that dominates much of the world of political science. The bottom line? The social sciences do a poor job of policing research — and the room for research that is propagandistic, incomplete and lacking in real-world context is enormous.

Unfortunately, the mainstream media is composed of individuals with little understanding of research, statistics or the culture of academia — and it thus utterly gullible when anything is presented by someone with a Ph.D. following their name. The result is a public presented with supposedly "unbiased" research that is anything but. In the view of the media, if a bunch of smart people say it, it must be true.

LaCour has mounted a desperate defense. He claims the lack of corroborating evidence is not his fault. LaCour claims his original data were "destroyed due to privacy/confidentiality concerns." Rubbish. One thing a researcher never does is destroy the original research. LaCour's "dog-ate-my-homework" alibi is just ridiculous. http://retractionwatch.com/2015/05/30/weekend-reads-gay-canvassing-study-saga-continues-elsevier-policy-sparks-concern-a-string-of-scandals/

The collapse of LaCour's house of cards and how it happened reveals three systemic problems in academia: 1) Conservative (!) groupthink; 2) Built-in incentive for fraud and 3) Opportunity for fraud.

First, It is no secret that academia is overwhelmingly left-leaning in its political mindset. https://www.insidehighered.com/news/2012/10/24/survey-finds-professors-already-liberal-have-moved-further-left The American Political Science Association is the last organization on Earth that takes Marxism seriously. But the real problem is a conservative groupthink of the left. Conservatism in this case is a resistance to change and a constant reinforcement of the existing order. Groupthink is, of course, single-minded thought with room only for carefully bounded dissent — but never any true dissent that would upset the established order.

Criticism is assiduously avoided. Not only that, but new researchers — just the sort of people most likely to bring forward— are in a particularly vulnerable position. In my own doctoral research, I found one article in a prestigious journal that was particularly terrible, yet I was told to be cautious in criticizing others in the field of earmark research, since one of them might be on a peer review panel.

LaCour's "breakthrough" research was just what the doctors ordered. It was a new revelation that supported their own preferred political project. It was the group that exposed LaCour's fraud who were subject to warnings about rocking the boat. Fortunately, the fraud of LaCour was so obvious and the evidence so damning, that debunking his study was impossible to avoid.

Second, advancement in academia can be summed up by the phrase "publish or perish." If you want tenure, new grants, prestige, you have to constantly churn out new research. Unfortunately, new insights are tough to find. Much of what we find in social science is maddeningly complex. Phenomena have multiple causes — and it's difficult to isolate what matters most. Prominence is no cure. LaCour's so-called "co-author," Dr. Donald Green, clearly did not significantly participate in the research. He was just window-dressing for LaCour with a little resume inflation for himself.

With all the pressure on new research, checking on others' research is devalued. There is little advancement and grant money for checking on the work of others. The "peer review" process, where others in the field review their colleagues work (for free), is hardly foolproof. Peer review is only as good as the seriousness with which the reviewers take it. Sometimes peer review is just a cursory read, with the reviewer making sure his/her research is cited. Simply put, there isn't much reward for being the police.

And, the social sciences are particularly vulnerable to fraud. Any social science research requires the researcher to make judgment calls on what data and variables to use. If you put together your own data, it can be subtly or overtly faked (like it appears Mr. LaCour did). If you are interviewing people, you can always change your notes or transcripts to suit your needs. You can even fake entire interviews.

In my own dissertation http://www.amazon.com/Washington-Gold-Rush-Competition-Congressional/dp/3659590835 on congressional earmarking, I had plenty of opportunities to manipulate data, alter interviews, and generally massage my evidence after the fact to bolster my conclusions. (For the record, I didn't!) And "improving" one's research does not necessarily require LaCour-style wholesale fraud. Sometimes a few subtle tweaks are all it takes to turn something from inconclusive to statistically significant.

The bottom line is that fraud happens and likely a lot more than we realize. The blog "Retraction Watch" is dedicated to exposing withdrawn research. http://retractionwatch.com/ Not only is Retraction Watch hardly starved for content, it only deals with formal retractions. Just like any prosecutor knows that just a small fraction of criminals are caught, Retraction Watch likely is just scratching the surface on bad research.

What academia needs is far more skepticism, policing and, most of all, punishment. The media need to stop accepting one-off studies as the new gospel truth. The world of academia needs to place much more emphasis on reviewing and vetting research and more rigorous standards of record-keeping need to be employed.

But, most of all, there must be consequences for fraud. While Mr. LaCour's academic career is over, his fraud was abetted by a wide circle. Dr. Green should suffer consequences more than momentary embarrassment for allowing his name to be attached to LaCour's article. Researchers like Green need to stop engaging in this academic promiscuity. Mr. LaCour's adviser at UCLA should have caught the obvious problems in his research. LaCour's home department, UCLA Political Science, is responsible for the quality of research produced by their students. Only if UCLA (and other universities that let fraudulent research out) loses grant funds will it take this problem seriously.


Share   Share

Featured Columnists
Featured Audio Links