"... and Mr. LaCour said he would produce a “definitive” accounting by the end of next week.
Please follow up this interesting and intriguing drama with another article.
Please follow up this interesting and intriguing drama with another article.
4
Just as some social scientists will spin or completely fudge studies to advance their careers, so too, some journalists will spin and even fudge for the same reason.
The articles drawing parallels between this failure and the Regnerus debacle also range from spin such as this piece to outright fabrication, easily found on the websites of social conservatives. The similarity is, in fact, almost non-existent. Though there is, apparently, an intriguing connection of both scandals to the University of Texas...
The articles drawing parallels between this failure and the Regnerus debacle also range from spin such as this piece to outright fabrication, easily found on the websites of social conservatives. The similarity is, in fact, almost non-existent. Though there is, apparently, an intriguing connection of both scandals to the University of Texas...
1
Trust of a fellow scientist's observation lies rooted at the core of science, with the only check being independent replication by another. And that is what happened here: the irregularities in the LaCour-Green study arose when independent researchers tried and failed to replicate LaCour and Green's results. While the alleged fraud is horrible and exceedingly distressing, its rapid detected just a few months later really is a testament to how science rights itself and moves forward. The tragedy, though, is that what is not checked is the gross harm to the public's perception of science, and the loss of trust from those not equipped to replicate an observation or approach themselves. Science persists but public support for it is horribly shaken.
15
This is so typical of what is going on in this country today. Pick some left wing cause, and then come up with a study that gives you the results that the left wants to justify their cause. Its so common these days that I'm surprise anybody bothered to question this evident fraud? Global Warming is a prime example. Or I guess its Climate Change now. In that case there are plenty of well known and very competent scientists, who don't necessarily question that the earth is getting warmer, but they do question the distortion and manipulation of data, or the lack of repeatable results. That apparently has become anathema and considered blasphemy. At least this shoddy piece of work didn't go unchallenged. Good Job!
5
Unfortunately the right wing doesn't even bother with studies. The right wing and their media metal minions just simply lie outright and the Great Unhinged believe every word that drips from their lips. Kudos to Karl Rove, the Pavlovian Rovian of the latter day monkeyshine shams.
27
The headline is *still* inaccurate. This wasn't a study of "gay canvassers." The study assessed a particular personalized, unscripted face-to-face canvassing technique, employed by both gay and straight canvassers. So the headline should be "Doubts About Study of Canvassing Method Rattles The Field." But the NYT really, really wants to get the word "gay" in the headline because it will drive clicks.
22
Exactly. I said the same thing early this morning. This is all about clickbait but it works because I've accessed the same article and commented around 5 times. If it wasn't a so-called "gay" study, I wouldn't have had any interest.
THAT'S the study that someone should consider. Have a newspaper article or Facebook page and use hot button words: "gay" "Blacks" "immigration" "abortion" "Obamacare," "Common Core" etc., etc. and see how many people click on the article, comment on it and how many times.
THAT'S the study that someone should consider. Have a newspaper article or Facebook page and use hot button words: "gay" "Blacks" "immigration" "abortion" "Obamacare," "Common Core" etc., etc. and see how many people click on the article, comment on it and how many times.
10
Something that has been overlooked is that most reputable scientific journals today require all the authors to sign a statement saying they had substantially participated in either the gathering of the data or the writing of the paper. This was instituted because senior academics would often place their names on the work of subordinates without doing any work.
One has to wonder what work Donald Green did on the paper. He clearly was not involved in the research and if he didn't even see the data it's difficult to see how he could have made a substantial contribution to the writing of the paper.
He should be held just as culpable for this debacle as Michael LaCour and as it would be a case of academic fraud I don't believe tenure would prevent Columbia from dismissing him for this.
One has to wonder what work Donald Green did on the paper. He clearly was not involved in the research and if he didn't even see the data it's difficult to see how he could have made a substantial contribution to the writing of the paper.
He should be held just as culpable for this debacle as Michael LaCour and as it would be a case of academic fraud I don't believe tenure would prevent Columbia from dismissing him for this.
24
We should settle for just ignoring polls completely. After all, they are too easily biased just by the wording of the questions.
3
The elephant in the room I think, is that what the study suggests, that people will change their minds with a one on one conversation is most probably true. From the beginnings when advocates for GLBT teachers in California where able to convince voters to not discriminate against openly gay teachers; to a reversal in Maine and victory in Minnesota and now Ireland. Advocates for equality used the same tactic: one on one conversations and "coming out" .... so the research is tainted obviously but the result is clear.
Kinda like paint by numbers dontcha think?
What did you expect from political science, which is about as scientific as voodoo? "Feeling thermometer"....a subjective measure masquerading as objective, quantitative data. Nasty trolls will now leap upon the errant scholar - one can't help but wonder if he's getting this much heat because he dared touch the tail of a current sacred cow. As to peer review, it often has the effect of ensuring only conformity.
18
I have been a physician for 40 years. I couldn't agree more with Jim's comment. Social science is mostly psychobabble, biased in many ways, and certainly not scientific.
2
where does Science Fiction fall in your hierarchy?
"Mr. LaCour told Dr. Green that the response rate was high because he was paying respondents to participate, a common and accepted practice."
How do you separate the effect of the payment from the effect of the interview? If some same-sex-marriage opponent was given a check for $1,000 from some gay group, that alone might be a sufficient incentive to change their position.
How do you separate the effect of the payment from the effect of the interview? If some same-sex-marriage opponent was given a check for $1,000 from some gay group, that alone might be a sufficient incentive to change their position.
9
This is a classic case of "advocacy science" where the results are coerced (or totally faked) to match the researcher's opinion. I believe the same thing happens regularly in global warming "research." Has the raw data ever been released to support the temperature forecast models? When questioned, most climate change researchers respond with "trust me, I'm an expert, climate change is an established fact, so you don't need to see any data."
11
Wrong, on almost every front.
First, yes, as a matter of fact the raw data is widely-available to reasearchers (not to mention that raw data is often not the best or most reliable data for analysis).
Second, almost no climate researchers say "trust me" regarding their data; they use publicly-available data or they publish their data so that it is public.
For example, the IPCC has an entire "data distribution" center that publishes a wide number of primary data sets.
So, once again, what we have is climate change denialists repeating errors (falsehoods?) prepared by the Heartland Institute or some other Koch-infested entity.
There is essentially no serious challenge to the consensus views of climate scientists in IPCC or elsewhere and there is an overwhelming quantity of data validating the models.
First, yes, as a matter of fact the raw data is widely-available to reasearchers (not to mention that raw data is often not the best or most reliable data for analysis).
Second, almost no climate researchers say "trust me" regarding their data; they use publicly-available data or they publish their data so that it is public.
For example, the IPCC has an entire "data distribution" center that publishes a wide number of primary data sets.
So, once again, what we have is climate change denialists repeating errors (falsehoods?) prepared by the Heartland Institute or some other Koch-infested entity.
There is essentially no serious challenge to the consensus views of climate scientists in IPCC or elsewhere and there is an overwhelming quantity of data validating the models.
16
A reference for your assertion?
1
Obviously you have not spent time or effort in reviewing the Climategate emails and files. There was no involvement of or by "Heartland Institute or some other Koch-infested entity" only government employees and government funded academics.
1
One underlying academic racket that people seem to miss here is the sleazy practice of faculty advisors putting their names on papers they did not write. There is nothing about being an advisor that that permits someone to take credit for the work of their student. To put your name on a paper when you haven't even seen the data, no less written anything but your signature, is academy is dishonesty. Green should be fired.
14
There seems little question that Mr. LaCour lied. That's bad.
But it also obscures one of the inconvenient truths about scientific research today. To get a good position, and to get grant money, is tremendously competitive. You need to publish a lot and do a lot of research. And that research needs to succeed, not fail.
Some people cheat to do that, like Mr. LaCour. But others, even though they don't cheat, shy away from doing anything risky, anything that might fail.
Problem is, if there is one thing we know about progress in science, success only comes after lots and lots of failure. When everyone is afraid to fail, no one succeeds.
So people crank out research only when they know what the answer is going to be. And science moves forward only in small, incremental steps.
We see this in cancer research. No breakthroughs have come for decades. And what almost all cancer researchers are working on will not be any better. Even if their research succeeds, it fails, in that no real progress is made.
No one is exploring the frontiers of science. It's too risky out there. And we're all the worse for that.
But it also obscures one of the inconvenient truths about scientific research today. To get a good position, and to get grant money, is tremendously competitive. You need to publish a lot and do a lot of research. And that research needs to succeed, not fail.
Some people cheat to do that, like Mr. LaCour. But others, even though they don't cheat, shy away from doing anything risky, anything that might fail.
Problem is, if there is one thing we know about progress in science, success only comes after lots and lots of failure. When everyone is afraid to fail, no one succeeds.
So people crank out research only when they know what the answer is going to be. And science moves forward only in small, incremental steps.
We see this in cancer research. No breakthroughs have come for decades. And what almost all cancer researchers are working on will not be any better. Even if their research succeeds, it fails, in that no real progress is made.
No one is exploring the frontiers of science. It's too risky out there. And we're all the worse for that.
19
"We see this in cancer research. No breakthroughs have come for decades."
This is a ridiculously ignorant comment. In the past few years there have been major breakthroughs obtained with antibodies targeting immune checkpoints like CTLA-4 and PD-1. And CD8 T cells gene-modified to eliminate B cell cancers. Google "cancer immunotherapy" and you should find some articles covering the progress in this area.
This progress has all derived from basic research supported by the National Institutes of Health (NIH), which has seen its funding decline in real terms in recent years thanks to politicians who seemingly struggle to grasp its importance.
This is a ridiculously ignorant comment. In the past few years there have been major breakthroughs obtained with antibodies targeting immune checkpoints like CTLA-4 and PD-1. And CD8 T cells gene-modified to eliminate B cell cancers. Google "cancer immunotherapy" and you should find some articles covering the progress in this area.
This progress has all derived from basic research supported by the National Institutes of Health (NIH), which has seen its funding decline in real terms in recent years thanks to politicians who seemingly struggle to grasp its importance.
11
There is not nor will there ever be any "science" in the social "sciences". From economics to politics to sociology they are all merely racial, gender, colored ethnic, sectarian human legal history plus arithmetic. Too many variables and unknowns to control for or repeat and replicate.
5
"Too many variables and unknowns to control for or repeat and replicate."
You're missing the point: This is "advocacy science" where the goal is to buttress a social position regardless of the truth.
You're missing the point: This is "advocacy science" where the goal is to buttress a social position regardless of the truth.
11
even after all the education Medical Doctors are still in a practice. So what kind of Science is that?
1
As Mark Twain noted, "Figures don't lie, but liars can figure!"
7
People are being unfairly harsh to Dr. Green. It's neither expected nor feasible for each co-author to review raw data from a particular study. Ideally, there would have been more people involved in the project who could have noticed these irregularities sooner, but disallowing research involving two authors from different locations would do more harm than good.
When a scholar is willing to simply make up data (as La Cour seems to have done) it's not easy for co-authors to detect. It's actually easier to detect after the fact, through failure to replicate and other oddities in the data (as it was in this case). This system works and thankfully, apart from the likely derailment of La Cour's career and the embarrassment for Science and Dr. Green, the harm here was mostly reversible.
When a scholar is willing to simply make up data (as La Cour seems to have done) it's not easy for co-authors to detect. It's actually easier to detect after the fact, through failure to replicate and other oddities in the data (as it was in this case). This system works and thankfully, apart from the likely derailment of La Cour's career and the embarrassment for Science and Dr. Green, the harm here was mostly reversible.
15
What if La Cour's findings were fake but accurate?
The fault lies with LaCour and only LaCour. It's his work, his name, and his lack of integrity. I'm a soon-to-be PhD student (if I get in) and I can't imagine trying to fake my way through such a project and lying to so many people. This is more than academic dishonesty from LaCour; it's more like a psychopathology.
25
Too complicated article for me to follow. Will this affect the existing same sex marriages in California? I hope not.
Now the question of scientific integrity is out in the open perhaps you should open the gigantic Pandora's box of scientific fraud that is called global warming,climate change, or whatever is the current designation. Having perused very carefully the Climategate emails and files there is little doubt that a a conspiracy existed then and probably continues. If you have not or chose not to study the Climategate information the emails and files exposed the extent of their efforts to "hide the decline" in global warming, to destroy correspondence to avoid complying with FOIA Requests, the inability of the CRU to reproduce the charts and graphs used to support the AGW theory, the confirmation that they not only controlled the papers that would be considered by the IPCC committees, but also, controlled which peer reviewed papers would be published in scientific publications to the point that they were able to have an editor fired for publishing paper that challenged the AGW consensus. Another example, the Wegman Commission's, appointed by Congress, report on the Mann "hockey stick" curve concluded, "in our further exploration of the social network of authorships in temperature reconstruction, "we found that at least 43 authors have direct ties to Dr. Mann by virtue of coauthored papers with him. There is no scientific integrity but a classic example of what Professor Richard Feynman described as "Cargo Cult Science", the practice of failing to provide all available results.
4
The US National Science Foundation funds about $300 million for Economics, Social Science and Politics. I hope the NSF monitors the results carefully.
What is the meaning of co-authorship if the co-author has never seen the data, did not know basic facts about the research protocol, and apparently never critically examined the study's conclusions? Is Green only interested in appending his name to studies, or is he interested in science?
32
In the last few years there have been a number of scandals in the field of social psychology. The most noted ones have involved rather major figures. These have also had the adverse effect of badly affecting the students of these people through no, or little, fault of the students.
The main ones reported have involved social psychologists working in the Netherlands (Stapel, Smeesters, Forsters). See e.g.
http://www.sciencemag.org/content/344/6184/566.full
I think these are even worse than the current situation. And at least in this case the problem appears to have been caught before the person went on to a position of major influence.
The main ones reported have involved social psychologists working in the Netherlands (Stapel, Smeesters, Forsters). See e.g.
http://www.sciencemag.org/content/344/6184/566.full
I think these are even worse than the current situation. And at least in this case the problem appears to have been caught before the person went on to a position of major influence.
3
I think the shocking matter is how easily this individual was able to manipulate the peer review system in order to produce results. Other questions could be why his adviser was not reviewing his materials, further if he was receiving such large donations under what account would this have been stored? Schools are generally very protective of their donations for funding studies, since schools will take a certain percentage. It would appear the gentlemen picked all the correct buttons to push to make this work. He gathered the backing of a highly respected scholar, picked a hot button topic, gathered results everyone was expecting and it seemed that everyone allowed this to flow.
7
Yes, sort of like how shockingly easy it is to manipulate bond rating agencies.
3
Take a breath. Fraud and manipulation of social science data, particularly data gathered in a field study, are hardly a novel discovery. In neighborhoods like those chosen for this field study, one can predict, based on socio-demographic (age, ethnicity, race, education) and cultural variables (viz.: religion, prescribed marriage patterns, anti/pro-choice), the likelihood that voters would vote for or against same-sex marriage (at baseline) and which of the specific variables have the strongest effect on altering baseline opinions. Door-to-door chats, one would expect, would have little effect on moving a no vote on same-sex marriage. We ask: Which variables are most malleable in altering core cultural behavior like marriage partner selection? Two methodologies, a field study corroborated by a predictive statistical study can help to interpret field study findings. Next question: Who most likely coached canvassers on gathering and altering data in the field, the most likely place where field data can be altered without scrutiny by major professors? What type of data were gathered in the field? What were modes of analysis of field data?
5
Why was Mr LaCour, a Masters-level statistician at UCLA and a "PhD" candidate there, not with a UCLE professor supervising his PhD?
==
When I was getting my PhD, Nobel Laureate David Baltimore got into trouble, lost his post as head of MIT's whitehead institute because he co-authored a paper with a scientist whose research was probably faked. The burden is upon all of us who do research to validate claims by our co-authors.
==
When I was getting my PhD, Nobel Laureate David Baltimore got into trouble, lost his post as head of MIT's whitehead institute because he co-authored a paper with a scientist whose research was probably faked. The burden is upon all of us who do research to validate claims by our co-authors.
13
you must have missed this part of the article:
He would need an influential partner to help frame, interpret and place into context his findings — to produce an authoritative scientific answer. And he went to one of the giants in the field, Donald P. Green, a Columbia University professor and co-author of a widely used text on field experiments.
He would need an influential partner to help frame, interpret and place into context his findings — to produce an authoritative scientific answer. And he went to one of the giants in the field, Donald P. Green, a Columbia University professor and co-author of a widely used text on field experiments.
2
We seem to live in an age of high-profile cheaters and imposters. In the past few years, individual sports stars, journalists, business CEOs, scientists, historians, and doctors have all been exposed for cutting corners to make themselves, their work, or their bottom line look better than it is. Is this the Daedalus effect or something else? I would really like to understand what leads already high-achieving people to risk so much for ... what? Celebrity fame? Fortune? Status?
10
I have the feeling the subject being studied may have affected the lack of proper oversight and review. People and institutions are falling all over themselves to pander to the gay community now to make up for years of neglect and abuse.
4
Shame on authors Carey and Bellucci for erroneously stating the title as "Study in Science on Gay Marriage" in your mobile update - this is about a flawed study on the use of cancassino as a tool for political campaigns.
Shame too, on those in charge of superishing this grad student - Columbia's Dr Green and his UCLA adviser...Dr Green more so for being a "giant in the field of transparency in social science research study.
Since when shold you, Dr Green, sacrifice truth over protecting a junior scholar ("delicate situation when senior scholar makes a move to look at junior scholar's data"). This is a sad commentry for you and the trustworthiness of your integrity as a senior scholar, not to mention the field you serve, and the trust in higher education more broadly.
Shame too, on those in charge of superishing this grad student - Columbia's Dr Green and his UCLA adviser...Dr Green more so for being a "giant in the field of transparency in social science research study.
Since when shold you, Dr Green, sacrifice truth over protecting a junior scholar ("delicate situation when senior scholar makes a move to look at junior scholar's data"). This is a sad commentry for you and the trustworthiness of your integrity as a senior scholar, not to mention the field you serve, and the trust in higher education more broadly.
3
Whether a canvasser has a personal stake or not, I assume anyone canvassing door to door about a cause is passionate about said cause while either attempting to persuade support for it or attempting to dissuade someone from supporting it. It's a very interesting proposition. Unless highly rigorous test methods are employed and analysis of the results thoroughly vetted, a study of this nature should be dismissed as flawed since any authors of this study or any other study, have a personal stake in the test findings being viewed favorably and credibly. Complicating this particular study, doubts about its credible methodology and results aside, is the hot button issue of "same sex marriage." This subject typically elicits extremely passionate emotions from both supporters and detractors, with less middle of the road sentiments. It is reasonable to question the motives of both supporters and detractors of this study for potential biases toward the subject matter itself of this "flawed" study, possibly fueled by their own respective personal stakes and lack of objectivity.
Having been a canvasser, I think a high degree of skepticism is in order about the ability of a canvasser to change all but those who minds are already on the verge of changing. I don't know why it mattered to the researcher that those with a stake in the outcome would be more effective. If anything, that makes canvassing even harder.
1
Some would use this as a reason to deny all science. First, unlike the physical sciences, social science depends greatly an the quality of its data, especially if it includes questions in which respondents will alter their answers to affect outcomes. Second, this was a new study using questionable methods etc. Typically no one accepts the outcome until it is duplicated, if not triple etc. We are still confirming Einsteins theories. Now most climate change models etc have had multiple studies over the same data, which is far more objective, and accurately measured. To use this as an indication of the unreliability of science means you don't understand how it works. In this case other researchers immediately jumped on the results etc. I.e., science.
4
As a researcher who has worked on grants and coauthored with a variety of people, I can confidently say that:
If a grad student says he has hundreds of thousands of dollars in grant money, I am not embarrassed to ask 'from where?'
And if I am coauthoring, I am not embarrassed to ask to read the grant proposal.
I truly do not understand the professional norms that are conveyed in this account of things.
If a grad student says he has hundreds of thousands of dollars in grant money, I am not embarrassed to ask 'from where?'
And if I am coauthoring, I am not embarrassed to ask to read the grant proposal.
I truly do not understand the professional norms that are conveyed in this account of things.
31
Take a look at medical journals. A first-prize winning book of the British Medical Association quotes: "Medical journals are an extension of the marketing arm of pharmaceutical companies." (from a paper by one of BMJ's former editors).
Here's one simple way these "scientific" journals are influenced. When a paper on a drug trial is submitted, a drug company may call and say that they will purchase $100,000 of reprints if the article is published. (It's sickening to think of these reprints used as sales tools later on by drug reps in doctors' offices.)
Advertising from pharmaceutical companies in medical journals is another science-wrecker. Lancet got 41% of it's revenues from ads and reprints. This is science?
Here's one simple way these "scientific" journals are influenced. When a paper on a drug trial is submitted, a drug company may call and say that they will purchase $100,000 of reprints if the article is published. (It's sickening to think of these reprints used as sales tools later on by drug reps in doctors' offices.)
Advertising from pharmaceutical companies in medical journals is another science-wrecker. Lancet got 41% of it's revenues from ads and reprints. This is science?
8
One reason why these results are improbable is that equal-marriage opponents tend nowadays to be religious conservatives, who are not susceptible to rational argument.
5
Actually your statement is per se an irrational argument, because you're suggesting that "religious conservatives" are incapable of reason, which is nonsense. You use the term "conservative" perjoratively, thus betraying your own misguided self-righteousness! The incredible success of the homosexual "rights movement" is based on deception (hiding the true purpose of the movement's goals, and the homosexual "community's" true lifestyles), manipulation (falsely equating the movement with 1960's-style racial and social justice issues), and sophistry (endless, juvenile arguments repleat with buzzwords like "bigotry" "fairness" "equality" "justice" ad nauseum). Those things, coupled with the Catholic Church's equally incredible loss of credibility and hypocriscy as evidenced by their own revolting homosexual/pedophile scandals over the last 40 years, and almost certainly far longer. The notion of same-sex marriage may sound like a good ideal to idealistic, easily manipulated (hence this article) college kids and women, but the rest of us are not fooled by the moronic appeals to "fairness" and "equality."
4
You think women are easily manipulated, Optimus? Think again.
Well, they get older but they don't grow up. The kindergarten mentality states that when the teacher leaves the room . . . BOOM! All hell breaks loose and you do whatever.
7
Regardless of the academic discipline, published research and the methods by which it is conducted needs to be subject to strict peer review and oversight by proven scholars in the field. Both ingredients were missing or suspect. We ask, "why?" Perhaps the issue of same-sex unions is so emotionally charged and politically driven that a rush to authenticate them, by whatever "scientific" means available, allowed for lax standards? Is a "feeling thermometer" a proven tool of scientific research? I would not rush to judgement, but suffice it to say that this is not the first time methodological questions have been raised about the veracity of research aimed at advocacy for same-sex unions. [Cf. John Boswell's 1980 study published by the Univ. of Chicago Press: "Christianity, Social Tolerance, and Homosexuality." It has become the sine quo non for those who would cite historical precedent for same-sex unions in Christianity, yet its methodology was seriously challenged by eminent biblical and theological scholars. (Eg. Richard B. Hays, now dean of Duke Univ. Divinity School.)] Inevitably, such questioning fades from view, eclipsed by the emotional/political atmosphere in which we live. As a fierce advocate of civil rights (including those of LGBTQ persons) I disagree with my church's conferring blessings that "imitate" Holy Matrimony without any theological justification. Am I alone in needing a more reliable method than a "feeling thermometer" to substantiate my views?
6
No sir, you are not alone but I fear you are no longer in the majority on this issue, nor am I. The issue I speak of is the calm and rational examination of the implementation of radical changes to our social structure and their consequences to future generations and to the structure itself. Questions need to be researched regarding how rank and file citizens could change their minds to quickly and thoroughly on an issue that had engendered opposite reactions a mere few years ago and upon which all of civilization had been in agreement for millennia.
So absolutely no one stopped to think about how manipulative was this study? How interesting that the ethics of the study did not seem to matter. What if the approach was to go to areas of the state that favored SSM and the approach was to persuade them to be against SSM. Would this study have ever made it through an ethics review? Of course not and that shows the bias that exists in our universities and with our professors. Shame on Dr. Green and Science journal. The shame on the student-fraud goes without saying.
12
Yes, equality of subject matter doesn't seem to be of concern to these "intellectuals," only subject matter whose outcome would support their already agreed-upon agendas and objectives. This is science by way of advocacy which, of course, is not science at all.
There's a simple fix to the problem: find a way to hold the institutions (in this case UCLA and Columbia, perhaps even Princeton) accountable for sloppy due diligence all around.
Also, if this guy's going to lose his job offer from Princeton, perhaps the senior Columbia guy should be given an equivalent punishment too.
Also, if this guy's going to lose his job offer from Princeton, perhaps the senior Columbia guy should be given an equivalent punishment too.
8
The latest fake-data scandal is not the first, or last: http://garysmithn.com/blog.html
3
“Given the negative publicity that has now surrounded this paper...."
It's mind boggling that "Science" cites negative publiicity as a primary argument for retracting a paper. That "publicity," positive or negative, should influence support for a published paper is antithetical to everything "Science" should, but apparently does not, stand for.
It's mind boggling that "Science" cites negative publiicity as a primary argument for retracting a paper. That "publicity," positive or negative, should influence support for a published paper is antithetical to everything "Science" should, but apparently does not, stand for.
11
You are correct. Today's "Science" in many fields relies less on adherence to the scientific method of hypothesis origination and testing (with the hypothesis being proven either true or false by the data) and more of the groupthink known as "science consensus". This is an aberration which many participants accept a hypothesis without confirmation via proper scientific methodology (the best example being "global warming") and the hypothesis is altered to fit the data or visa versa. This flawed research is certainly a result of "Science consensus" sprung from political correctness and personal ambition, none of which have a place in TRUE science.
Rather than having its faith in science shaken, this episode should have the opposite effect: it should affirm the public’s confidence in science as a self-correcting enterprise. Many incidents have demonstrated that major scientific findings are never hailed by other scientists. Instead, they are met with the greatest degree of skepticism. The most famous such case was cold fusion. More recently, what would have been a landmark study that claimed to have detected gravity waves was found to be inaccurate. It was the intense scrutiny that ensued, because this research was so important, that revealed the researchers’ error. In that case, there was not the least intent to mislead. It was a case of the researchers misinterpreting their data. These cases are not limited to the physical science either. In 2011, EO Wilson abandoned evolutionary psychology theory that he himself helped develop, saying the mathematical models upon which it is based no longer withstood scrutiny. Of course there are unscrupulous and sloppy scientists and publishers out there. But their deceit and mistakes inevitably get discovered.
20
Not really. "The Methods of Science" are not responsible for correcting this error. There was no "immense scrutiny" that followed the paper's publications. Nay, what happened is that the guy who liked the results and wanted to replicate them had the integrity to come forward and say that he could not replicate them. "Science" and "scientific skepticism" did not save the day. An honest man did.
1
Kyle J of Houston, what you describe *is* "the Methods of Science" at work. Possibly the most important aspect of scientific work is replication. When a study is published with a conclusive finding, the first reaction of The Scientist is to replicate the study and conpare results to either reaffirm the findings, or to reveal flaws within the previous study. In this case, that crucial step of replication was key in unraveling the fraud. Even if the replicator of the study had not contacted Dr. Green or the surveying company, the "Methods of Science" would have called for the second study to be published, revealing that the findings of the prior study do not apply in all cases. This would lead to "scrutiny" of both of these studies, and further replication, which would all show that the original study was in some way flawed. That's how it's supposed to work.
2
It also raises the question of publication bias. Would his results have been published -- would he have been offered a position at Princeton -- if he'd concluded that having a GLBT questioner didn't affect the responses of those being interviewed? I suspect not. I suspect he needed to get the result he wanted in order to get the job.
10
This is progress? Social, political or "scientific?" From decades ago having your name and address published, losing your job, career and family, to today?
It ain't journalism either.
It ain't journalism either.
2
Was Mr. LaCour's research part of his Ph.D. work? If so, why is there not more focus on the (lack of) oversight from his advisor, Professor Lynn Vavreck? And, by the way, Professor Vavreck is identified in the article as an associate professor but the UCLA Political Science website (http://www.polisci.ucla.edu/people/lynn-vavreck) identifies her as a "*full* professor of political science and communication studies at UCLA and a contributing columnist to The New York Times" (emphasis added).
9
I would not read too much into the Associate/Full Professor distinction. Both Associate and full Professors are tenured. Assistant Professors are not. It seems a reportorial inaccuracy, but it is also possible that when the article was being researched, Prof. Vavreck had not yet been promoted to full professorship or that her promotion was to take effect the following academic year, as is always the case. You come up for promotion, you get it, but the new rank is effective the following year, not the current year.
1
This story illustrates most of the things that can go wrong in a research project, the most important being the researcher's personal stake in the results. There were lots of other failures, the worst being the student's supervisor's distance from the work and his failure to insist on knowing the details. It's not a 'delicate matter' to ask for details, it is essential! In science, whether it is social science or physics, details are everything. The primary data were 'deleted' ? Give me a break! They probably never existed. If this student gets a degree for this work, the disgrace will be complete.
28
Dr. Green couldn't wait to sign up for this paper because its results were so stunning that he knew it would be published in a big time journal.
Because he did not even exercise even the most basic due diligence regarding his co-author, Dr. Green's crediblity and that of Science are now justifiably under scrutiny and question.
In this age of "choose your own science to suit your lifestyle" the lack of rigor and examination prior to publication is stupifyingly transparent.
Because he did not even exercise even the most basic due diligence regarding his co-author, Dr. Green's crediblity and that of Science are now justifiably under scrutiny and question.
In this age of "choose your own science to suit your lifestyle" the lack of rigor and examination prior to publication is stupifyingly transparent.
8
This American Life did a story on this study when everyone thought it was real and now that it appears to have problems, they have a much better description about at least some parts of the current situation than the New York Times has in this article.
3
To repeat what has been said here:
(1) Studies in the "soft" sciences are notorious for their unrepeatability - the acid test for scientific research.
(2) This is exactly what is supposed to happen to any kind of scientific research. Publish the paper and be prepared for the review process to work. Twenty years ago I remember a brouhaha in physics relative to "controlled fusion." It was nonsense - and that was in the realm of real science.
Thirty years of international economic experience and experiment has not been able to convince most people that "trickle down" is a synonym for "the rich get richer" etc. et. al.
(1) Studies in the "soft" sciences are notorious for their unrepeatability - the acid test for scientific research.
(2) This is exactly what is supposed to happen to any kind of scientific research. Publish the paper and be prepared for the review process to work. Twenty years ago I remember a brouhaha in physics relative to "controlled fusion." It was nonsense - and that was in the realm of real science.
Thirty years of international economic experience and experiment has not been able to convince most people that "trickle down" is a synonym for "the rich get richer" etc. et. al.
7
Thank you NYT's for demonstrating why it is so painfully difficult if not impossible to do science on anything that has to do with homosexuality. This morning I began an article titled "Doubts about study of gay unions....". I am have finished one called "Doubts about study of gay canvassers....", the difference is not trifiling and in kowtowing to one person's emotional frailty you have rendered your own article fairly baffeling. There was no study of gay canvessers, there was a review of a study conducted by gay gay canvassers and authored by a gay grad student. For god sake the original title could hardly be called a perjortaive appraisal of gay marriage. Is it really that hard to see that in creating one persons "safe space" you have dismembered everyone elses forum for learning?
7
In all this reporting, I'm not so sure that it is "fraud", but I see strong evidence of an LGBT echo chamber.
5
The big loser here is the falling reputation of "Science". Over the past years Science has all to often been lax in its vetting process, see, for example, the years-long debacle over articles on the Dino-Bird controversy.
7
I changed my opinion of Science when they added 'Magazine' to the title. A magazine is not a journal.
It's ironic this article comes right after last week's about the carelessness of science and the number of retractions on the rise in the social and medical sciences.
It appears our highly competitive society, and cavalier "get away with anything you can" attitude are combining to erode the integrity of studies, which is a shame. I think the bar has been lowered so low, in all realms of life in terms of entertainment, sports, academia, and business--as people realize only a few get "caught" and have to pay the price.
It's a shame because the one of the priceless things a man or woman truly has is honor--and once that's lost, it's pretty hard to get it back.
It appears our highly competitive society, and cavalier "get away with anything you can" attitude are combining to erode the integrity of studies, which is a shame. I think the bar has been lowered so low, in all realms of life in terms of entertainment, sports, academia, and business--as people realize only a few get "caught" and have to pay the price.
It's a shame because the one of the priceless things a man or woman truly has is honor--and once that's lost, it's pretty hard to get it back.
5
I think they all missed the "What Happens Next" lesson on Sesame Street. Too bad . . .
I don't see a case for blaming Dr. Green or any other advisor to Mr. LaCour. Sure, they could have exerted a little more oversight, but this seems like a case of pure fraud by one person, Mr. LaCour. There's usually little that can be done in science about falsification by one person. I'm impressed with the eventual skepticism shown by those around Mr. LaCour.
8
When I read this I immediately thought of the global warming scandals, where they not only hid the data, but changed it to match their preconceived preferences. Where is the NYT's expose' on that? Oh, that's right, we are not allowed to question global warming.
8
Global warming and same sex rights...two issues which we are no longer allowed to debate or even question in the land of the free.
What happened here is an inevitable result of the cutthroat competition for positions in academia. Simply doing good work is not sufficient for a successful academic career; an aspiring academic must do something really noteworthy that is published in a prestigious journal. Hence, there's a large incentive to do the flashiest work possible. The alternative is starting a new career.
Those who think that this type of behavior is limited to the social sciences are delusional and may want to review the announcement of the detection of gravitational waves.
Those who think that this type of behavior is limited to the social sciences are delusional and may want to review the announcement of the detection of gravitational waves.
6
Shocking, I know, but scientists can be just as dogmatic as religious people and statistics can be used as bludgeons just as much as words.
6
"Over the weekend, the three of them, with the help of an assistant professor at Yale, Peter Aronow, discovered that statistical manipulations could easily have accounted for the findings"
So you can lie with statistics. What else is new?
So you can lie with statistics. What else is new?
7
This is why major journals in my field (evolutionary biology) have started requiring that raw data be uploaded to a public data archive.
10
Prof. Lynn Vavreck's UCLA faculty profile @
http://www.polisci.ucla.edu/people/lynn-vavreck
Lynn Vavreck is a full professor of political science and communication studies at UCLA and a contributing columnist to The New York Times. She teaches courses on and writes about campaigns, elections, and public opinion. She holds an M.S. and Ph.D. in political science from the University of Rochester and has previously worked at Princeton University, Dartmouth College, and The White House. She is the recipient of multiple grants from the National Science Foundation and awards for her work on political advertising. She has published four books, including The Message Matters, which Stanley Greenberg called “required reading” for presidential candidates, and The Gamble, described by Nate Silver as the “definitive account” of the 2012 election. Professor Vavreck has served on the advisory boards of the British and American National Election Studies and is the co-founder of the Cooperative Campaign Analysis Project. In 2014, she hosted Hillary Clinton at UCLA’s Luskin Lecture on Thought Leadership.
http://www.polisci.ucla.edu/people/lynn-vavreck
Lynn Vavreck is a full professor of political science and communication studies at UCLA and a contributing columnist to The New York Times. She teaches courses on and writes about campaigns, elections, and public opinion. She holds an M.S. and Ph.D. in political science from the University of Rochester and has previously worked at Princeton University, Dartmouth College, and The White House. She is the recipient of multiple grants from the National Science Foundation and awards for her work on political advertising. She has published four books, including The Message Matters, which Stanley Greenberg called “required reading” for presidential candidates, and The Gamble, described by Nate Silver as the “definitive account” of the 2012 election. Professor Vavreck has served on the advisory boards of the British and American National Election Studies and is the co-founder of the Cooperative Campaign Analysis Project. In 2014, she hosted Hillary Clinton at UCLA’s Luskin Lecture on Thought Leadership.
2
I heard about this study a few weeks ago on NPR. I couldn't believe then, and I still can't believe now, that any graduate student could possibly find funding for a study with such obvious flaws... unless it was designed in order to purposely lead respondents down a garden path.
Hasn't anyone in Dr. Green's lab or at Science ever heard of "social desirability bias"... or the tendency of the respondent to tell the researcher what s/he apparently wants to hear? The troubles about "missing raw data" and secret identity for the funding source/s only compound the ridiculousness.
Hasn't anyone in Dr. Green's lab or at Science ever heard of "social desirability bias"... or the tendency of the respondent to tell the researcher what s/he apparently wants to hear? The troubles about "missing raw data" and secret identity for the funding source/s only compound the ridiculousness.
6
I agree, but it shows the gullibility of people when they want to believe something is true to fit their view of how the world should run. Green should have asked several obvious questions including as the first question, "Why aren't you working with someone at your school of study instead of asking me to be your co-author?"
For a study of a subject area that is bound to be controversial, no matter what the results are--supported by a senior professor who wrote the textbook on field survey methods and analysis of those results, it is dismaying to hear that there was so little oversight. This is a question that it would really be nice to know the answer to, but sadly, we clearly won't get it from this study. And speaking of bound to be controversial, it is fascinating that some commenters think the money came from people who wanted to sabotage the study and other commenters think it must have come from advocates intent on promoting their agenda.
1
Here in California a few years ago a canvasser came to my door and asked for a donation to support marriage equality. I wrote out a check for $10. When I got my statement the next month the check had been altered to $100 (he actually added a 0).
I called the national headquarters of the organization and they would not guarantee me that the canvasser would be fired.
I was and still am I big supporter of marriage equality but, I must say, this particular organization disappointed me. The people on the front lines of any organization going door to door and raising money are the face of the cause. Fraudsters should be fired immediately.
I called the national headquarters of the organization and they would not guarantee me that the canvasser would be fired.
I was and still am I big supporter of marriage equality but, I must say, this particular organization disappointed me. The people on the front lines of any organization going door to door and raising money are the face of the cause. Fraudsters should be fired immediately.
6
You wrote the word "ten" and even with the discrepancy, the bank cashed it?
There's a reason checks have the amount written out as well as with digits.
There's a reason checks have the amount written out as well as with digits.
8
Are you sure it was the canvasser? It could have been whoever it was who deposited the checks, which I assume were made out to the organization, not the canvasser personally.
1
Gee, I wonder how many "Climate Change" studies would hold up under the modest amount scrutiny this guy's study got? What does this say about Ivy League Universities and the REAL quality of education therein since this guy suckered them into giving him a job because all they apparently care about was how much he published? And, finally, where are the "science is GOD" crowd who deny that they are, in the end, simply reliant on their FAITH in human beings (aka scientists, their reviewers and publishers) rather than a creator?
13
Gee, I wonder what area of scientific investigation has been more thoroughly and intensively scrutinized than climate science. And since you are obviously a climate change skeptic, please explain how your faith in God has led you to conclude that climate change is not occurring or is not human-caused.
9
Ah, another sad and clueless posting from someone who clearly has no clue of how science works. First, don't confuse social studies, which are in question here, with actual science. Long term cheating in real (hard) science is impossible, because nothing is regarded as established until it is reproduced several times by independent investigators. FAITH, as you put it, has absolutely nothing to do with anything. Actually, speaking as a research chemist, we like nothing better than to prove something wrong. This always opens up new questions and more research. Wouldn't even you admit that this is the precise opposite of the religious approach? Shalom.
12
Thanks for demonstrating a profound understanding of the difference between surveys and hard data collections.
6
*delicate* .... *difficult* are code words used by people who do not want to hold others accountable for their integrity, but rather are operating on their views about trust and other factors that attract people to one another based upon personal judgement. It is peer review and whether or not others can replicate an approach that is what matters, i.e. in reality and not just what we would like to believe.
10
I don't necessarily see Dr. Green's use of these two words as "code" as you do. In the academic culture of most research universities, his viewpoints are a cultural norm. There is an element of trust particularly if one is working with a bright, talented, student. My doctoral advisor never looked at my data. As an academic myself, I only ask to see raw data with less than stellar advisees and with those who claim results that are very, very extreme. This happens rarely. However, even then, I first look at the soundness of their data analysis, look for errors in the data analysis. Here is usually the answer. In sum, I don't fault Dr. Green, but I think the sad result is that he will be less willing to mentor what seems to be a promising grad student. And I don't blame him.
Here's the basic problem, and it is blindingly obvious:
Having a "Passionate" "Advocate" "Researcher.
Especially in a field like sociology where there are no objective and quantitative outcome measurements.
Having a "Passionate" "Advocate" "Researcher.
Especially in a field like sociology where there are no objective and quantitative outcome measurements.
8
15% is a good response rate? That alone should have precluded the study's publication.
8
@Jeden [for some reason my replies have not been showing up under the original comments]: The response rate has nothing to do with whether the data were meaningful. What is important is the number of responses and whether the results were statistically significant or not.
It is sad/angering that this was done, but it is refreshing to find a publisher who is appropriately dismayed by such chicanery. Contrast Science's reaction with the absolute lack of consequences for anyone involved in the Rolling Stone's disgraceful episode (completely discredited piece on alleged rape at UVA). Thank you, Science and Dr. McNutt. There is still time for you to lead in coming up with a silver lining to this, in the form of new standards for review procedures and the availability of data to reviewers.
5
Social science today is rife with questionable results that are tainted either by confirmation bias or only published if the results fit a certain narrative. Stray from the message and you won't get published or funded in the future. Sad and a disservice.
5
Self selection bias and the fact that participants might have been paid would tend to skew the results as well. There are questions concerning other studies' results validity due to the fact that participants were sought by posting fliers in gay nightclubs. But I also have to assume that both sides are suspect. You can have just as much fraud from opponents of gay marriage. Maybe some social phenomena just can't be studied because there is too much diversity among the pros and cons sides and too much emotional/religious baggage on both sides to really come away feeling that you were given an honest assessment of what drives peoples' motivations to be for or against a particular concept.
The premise that you can change a person's mind in a 20 minute conversation is flawed in that it would matter greatly which people you are attempting to influence. The questions and conversations would have to be constructed in such a way as to ensure that the participants are not being manipulated by the messenger but only by the content of the message.
This was ambitious for sure but literally impossible to carry out. The highly competitive job market is only going to tempt other graduate students to try their luck at perpetrating more of this type of fraud.
The premise that you can change a person's mind in a 20 minute conversation is flawed in that it would matter greatly which people you are attempting to influence. The questions and conversations would have to be constructed in such a way as to ensure that the participants are not being manipulated by the messenger but only by the content of the message.
This was ambitious for sure but literally impossible to carry out. The highly competitive job market is only going to tempt other graduate students to try their luck at perpetrating more of this type of fraud.
Political science, like military intelligence, is an oxymoron.
8
At least the Times has changed the headline from "gay unions" to "gay canvassers" but again, the issue is between one graduate student, his adviser and a journal. The study is about VOTING.
If you were to replace "gay" with "Black" and take the wayback machine to 1961, some readers would nod their heads and say see, this is why we shouldn't mix the races.
If you were to replace "gay" with "Black" and take the wayback machine to 1961, some readers would nod their heads and say see, this is why we shouldn't mix the races.
5
So what I'm saying here is that this is a clumsy article which, in the end, will be forgotten and everyone involved will continue to make money (including the researcher, who may get a book deal out of his inevitable mea culpa when it occurs) The LGBT community may wind up being hurt because of it, which is why the Times did no one any favors by their strange headlines.
Fraud and selective misrepresentation, as well as the publication of studies that do little but add to cv's is rampant and should come as no surprise. Every so often, all this is brought to the public's attention in a particularly egregious way.
But as a former reviewer for several journals in the behavioral sciences, I was astonished by some of the garbage I was given to review, and by the well done studies that said nothing.
But as a former reviewer for several journals in the behavioral sciences, I was astonished by some of the garbage I was given to review, and by the well done studies that said nothing.
38
its like the nilsson album "the point"-
"you see what you want to see
and you hear what you want to hear"
"you see what you want to see
and you hear what you want to hear"
3
Read with a slow head shake: Dr. Green, Dr. Green, Dr. Green.
About LaCour's proposed study: “I thought it was a very ambitious idea, so ambitious that it might not be suitable for a graduate student,” said Dr. Green.
So Dr. Green knew from the very start that his advice and guidance would be needed. And yet it appears from the NYT article that Green thoroughly abdicated his responsibility, but only until questions began to fly about the study's validity. Then Dr. Green deftly stepped in and claimed distance and concern for questioning his colleague's professional standing as the reason for not DOING HIS JOB.
It doesn't matter that reviewing doesn't pay. It does matter that Dr. Green's lack of action to hold LaCour accountable reflects horribly on Dr. Green's professional laurels, and, by association, UCLA's reputation.
The greatest responsibility here is to be born by Dr. Green and then Mr. LaCour. This is not a 50-50 split in responsibility. It is 70-30 at minimum.
I do not fault Science or its editor(s). They took Dr. Green at his word that he advised LaCour and reviewed the paper. Dr. Green, you see, in essence lied to Science when the paper was submitted for publication. He did not fulfill his obligation as a co-author in any way.
If I were in UCLA administration (GO BRUINS!) I would form an independent committee that would review Dr. Green's papers and co-authored papers and see where else he may have abdicated responsibility.
About LaCour's proposed study: “I thought it was a very ambitious idea, so ambitious that it might not be suitable for a graduate student,” said Dr. Green.
So Dr. Green knew from the very start that his advice and guidance would be needed. And yet it appears from the NYT article that Green thoroughly abdicated his responsibility, but only until questions began to fly about the study's validity. Then Dr. Green deftly stepped in and claimed distance and concern for questioning his colleague's professional standing as the reason for not DOING HIS JOB.
It doesn't matter that reviewing doesn't pay. It does matter that Dr. Green's lack of action to hold LaCour accountable reflects horribly on Dr. Green's professional laurels, and, by association, UCLA's reputation.
The greatest responsibility here is to be born by Dr. Green and then Mr. LaCour. This is not a 50-50 split in responsibility. It is 70-30 at minimum.
I do not fault Science or its editor(s). They took Dr. Green at his word that he advised LaCour and reviewed the paper. Dr. Green, you see, in essence lied to Science when the paper was submitted for publication. He did not fulfill his obligation as a co-author in any way.
If I were in UCLA administration (GO BRUINS!) I would form an independent committee that would review Dr. Green's papers and co-authored papers and see where else he may have abdicated responsibility.
12
Dr. Green doesn't work for UCLA. He works for Columbia.
5
FYI, Dr. Green is at Columbia, not UCLA. I agree with you that Dr. Green was remiss in every way. Dr. Green says “It’s a very delicate situation when a senior scholar makes a move to look at a junior scholar’s data set,” but it was his JOB to look at the data. Dr. Green definitely had an early stage "heads up" -- Mr. LaCour's refusal to post the raw data -- that he inexplicably chose to ignore.
4
Wrong on multiple counts--including the fact that Dr. Green isn't at UCLA. He didn't abdicate responsibility; he expected his co-author to have integrity and to be working in good faith. Science pretty much depends on all of us acting in good faith, and most of us do. Unfortunately, someone who is incredibly unethical can sometimes try to take advantage of that, but pretending that the person who was lied to is at fault for believing the lie, rather than blaming the liar, is ridiculous. I guess we'll all have to be a bit more cautious about our co-authors now, but I don't think it's realistic to expect everyone to act as if they think they are being lied to constantly.
5
Boy Howdy! How long did it take to expose the obvious. Polls and sats are directed by the questions, how the question is phrased and who's asking the question. Makes you wonder how many issues were decided by false reading of polls.
9
This now-defrocked scholar told people something they wanted to hear: he was widely applauded for it and ultimately rewarded for it with a plum job at Princeton.
The question it brings up for me is: how do we treat scholars that tell us things we *don't* want to hear, but which may nonetheless be true?
The question it brings up for me is: how do we treat scholars that tell us things we *don't* want to hear, but which may nonetheless be true?
44
He has not been given the job at Princeton, and after this, may never be.
4
It happens all the time. Researcher A presents a thesis, and Researcher B rebuts it. If it's newsworthy more people hear about it, but generally the academic discussion is relegated to scholarly publications within ts field.
The major redefinition of a central institution of our species is happening at the speed of internet communications, It is no surprise that this social science research is flawed, as this movement has been driven by social contagion, with all serious research avoided once the movement took hold.
Here's an example from my correspondence on an OpEd in this publication that happened to be printed on the day of oral arguments in the Supreme Court case that struck down DOMA:
http://alrodbell.blogspot.com/2013_03_01_archive.html
Here's an example from my correspondence on an OpEd in this publication that happened to be printed on the day of oral arguments in the Supreme Court case that struck down DOMA:
http://alrodbell.blogspot.com/2013_03_01_archive.html
What is Science (the publication) doing publishing on social science (as opposed to biochem, cell, molecular, immunology, maybe physical sciences - ya know HARD science)??
Seems really bizarre that this faulty study was even submitted to Science in the first place.
Seems really bizarre that this faulty study was even submitted to Science in the first place.
15
@reddirt [for some reason my replies have not been showing up under the original comments]: Science is not limited only to the natural sciences, and I don't think it ever has been. It is the flagship publication of the AAAS, which is a general-science organization and has an entire section (Section K) devoted to the social, economic, and political sciences. Ya know?
Your Headline is as ridiculous as "falsified data" - please change it.
4
This is of course a joke. But one the well suits the discipline in which it is unfolding. A senior political scientist who has, one must assume, studied the political behavior of humans beings for many years sponsors an elaborate study by an ambitious young aspirant. He is reluctant to endorse a project of the scope proposed but agrees to and becomes a co-author.
Where did the funding come from? Never asked.
Where is the data? The computer ate it.
The company that was said to run the survey? Never heard of the study they were supposed to have conducted
In the old day, mid-1970's, at Columbia University in another social science, I designed the test stimuli and conducted pilot studies, I conducted to study itself with hundred of individual human respondents, I recorded the data, I punched the computer cards, I ran the statistical programs, I wrote every word of my dissertation and a friend typed the final version. I was scandalized to learn that other graduate students hired people to do all of these things.
Now we do better than that. The truly managerial can simply make it up sometimes get over.
Where did the funding come from? Never asked.
Where is the data? The computer ate it.
The company that was said to run the survey? Never heard of the study they were supposed to have conducted
In the old day, mid-1970's, at Columbia University in another social science, I designed the test stimuli and conducted pilot studies, I conducted to study itself with hundred of individual human respondents, I recorded the data, I punched the computer cards, I ran the statistical programs, I wrote every word of my dissertation and a friend typed the final version. I was scandalized to learn that other graduate students hired people to do all of these things.
Now we do better than that. The truly managerial can simply make it up sometimes get over.
9
I got my PhD in 73 in US History from Columbia, and while I know there were even then some academic grant hustlers (as a postdoc I even worked for one such "think tank"), speaking for myself and for most of the grad students I knew, I honestly don't think it would have occurred to any of us to willfully invent or even fudge data.
One hates to be like the fusty old grandpa, bemoaning the modern decline of standards, but really how else does one account for the almost daily and across the board litany of such, in a favorite lament of an old Columbia friend of mine, "snares, frauds and delusions?"
One hates to be like the fusty old grandpa, bemoaning the modern decline of standards, but really how else does one account for the almost daily and across the board litany of such, in a favorite lament of an old Columbia friend of mine, "snares, frauds and delusions?"
1
I do agree that the turn to co-authoring in poli sci over the last decade or so makes this more likely. It means that there are often co-authors who aren't particularly familiar with the data or the methods (they may have plenty to do in the division of labor, just not that) involved in a study. I don't think it makes for better political analysis...but is one of the consequences of a publish or perish environment and an obsession if ever more esoteric methods (the methods co-author may not know anything about actual politics!).
The only reason this is a story is because Liberals and the gay community didn't like the results of this study. If the same study conditions produced more favorable results there wouldn't be this hoopla. Where was the outrage when scientists in Europe were caught doctoring global warming data? It's completely believable that pounding on people's doors would make people NOT support your cause. These "canvassers" should really be called "punishment for being at home".
7
I don't think you read the article.
2
Michael LaCour’s inability to provide raw data and a clear explanation of his methodology has not only tarnished his reputation but also exposes the unprofessionalism of Dr. Green and Dr. McNulty.
How many other scientific studies have Green and McNulty approved without verification? LaCour has damaged the reputation of not only himself but his faculty advisor Lynn Vavreck and his sloppiness now calls into question the past and future work of Green and McNulty.
Fortunately the academic community is large enough that a few of their peers were able to challenge the findings of the Green/LaCour report. Even so, it makes one wonder how often statistics are fudged and raw data fabricated in any so-called scientific study. It will be interesting to learn if UCLA will grant LaCour his Ph.D. and if he will ever be employed in academia.
How many other scientific studies have Green and McNulty approved without verification? LaCour has damaged the reputation of not only himself but his faculty advisor Lynn Vavreck and his sloppiness now calls into question the past and future work of Green and McNulty.
Fortunately the academic community is large enough that a few of their peers were able to challenge the findings of the Green/LaCour report. Even so, it makes one wonder how often statistics are fudged and raw data fabricated in any so-called scientific study. It will be interesting to learn if UCLA will grant LaCour his Ph.D. and if he will ever be employed in academia.
37
This reminds me of another "fudging" after the Obama - Romney Presidential Debate. I watched the debate and the President was clearly in charge while Romney was red-faced and angry. We were told that Romney "won the debate" in order for people to get excited about the rematch - the next debate. This meme was repeated over and over. Chris Matthews nearly had a heart attack on MSNBC that night, complaining about his beloved President.
Even Obama conceded that he lost the debate.
Except that he didn't and most voters knew it.
And what does a "debate" mean, anyway?
Even Obama conceded that he lost the debate.
Except that he didn't and most voters knew it.
And what does a "debate" mean, anyway?
3
Other than a sloppy study, it doesn't change the fact that people are the ways they are by virtue of "laws of nature and of Nature's God." We are all bisexual said Carl Jung.
Why is the LGBT community not also including Queers, the Intersexed, the Asexual and Unisexual?
All are entitled to their constitutional rights. A sloppy social studies study should not be grounds for denying citizens of their constitutional rights.
Why is the LGBT community not also including Queers, the Intersexed, the Asexual and Unisexual?
All are entitled to their constitutional rights. A sloppy social studies study should not be grounds for denying citizens of their constitutional rights.
2
I am a senior social scientist at the University of Arizona (45 years on the faculty). Dr. Green is wrong when he claims that it is "delicate" for a senior scholar to look into a graduate student and co-author's data set or ask who funded the study. This is routine, standard practice among my colleagues. It raises questions about Dr. Green's competence, ethics, and a close inspection of his "over 100" publications.
33
Yes, the peer review process is broken; and has been for some years.
I can tell you, quickly, exactly why and how.
A PhD is supposed to mean the person holding it has been "Trained" in the rigors of science and scholarship, AND - has been "Certified" competent by other competent scholars.
The truth is that now, with the vast proliferation of information, and increasing specialization - the PhD degree is awarded without training in scholarship/science per se, and with automatic certification based on endurance, not mental capacity.
Crank 'em out. Big Multi Nationals are waiting to hire them.
And here we are. On a planet totally dominated by a species that is utterly reliant on "technology" - with people overseeing that technology - who do not know what they are doing.
Good luck to us. We need it.
I can tell you, quickly, exactly why and how.
A PhD is supposed to mean the person holding it has been "Trained" in the rigors of science and scholarship, AND - has been "Certified" competent by other competent scholars.
The truth is that now, with the vast proliferation of information, and increasing specialization - the PhD degree is awarded without training in scholarship/science per se, and with automatic certification based on endurance, not mental capacity.
Crank 'em out. Big Multi Nationals are waiting to hire them.
And here we are. On a planet totally dominated by a species that is utterly reliant on "technology" - with people overseeing that technology - who do not know what they are doing.
Good luck to us. We need it.
6
Actually, we crank them out, and most remain underemployed. There aren't lots of people ready to hire them...which is part of the problem. Too many grad students, too few jobs, so they are super competitive for a few jobs--apparently, this incentivizes faking one's data for some.
1
Why isn't Lynn Vavreck, his dissertation advisor, named at all in this article? Is she that media savvy to wriggle her way out of this? I am a PhD student, and your advisor is the one supposed to keep tabs on you, and to certainly catch inconsistencies in your CV (half a million dollars in grants?). She is just as negligent as Don Green appears in this article. I would definitely like to hear from her.
28
Last sentence, 3rd paragraph from last:
'Dr. Green called Mr. LaCour’s academic adviser, Lynn Vavreck, an associate professor, who confronted Mr. LaCour.'
(as printed in article online at 9am PST)
'Dr. Green called Mr. LaCour’s academic adviser, Lynn Vavreck, an associate professor, who confronted Mr. LaCour.'
(as printed in article online at 9am PST)
analysis and called their former instructor, Dr. Green. Over the weekend, the three of them, with the help of an assistant professor at Yale, Peter Aronow, discovered that statistical manipulations could easily have accounted for the findings. Dr. Green called Mr. LaCour’s academic adviser, Lynn Vavreck, an associate professor, who confronted Mr. LaCour."
Ok, she's named, but is portrayed as being completely inculpable. I'd like to see some blame shared her way.
1
What this article actually describes is a basic tenet of the scientific method working yet again--independent verification. Independent verification identifies mistakes regardless of whether they are honest or deliberate (e.g. see XMRV and chronic fatigue syndrome). But this NYT piece doesn't even acknowledge this fact, instead going for a load of breathless misleading nonsense about shaken public trust. It's fully understood by scientists that peer review cannot catch every mistake (or the worst case scenario of deliberate manipulation). That's where independent verification comes in.
The fact that the media is willing to trumpet results that haven't been independently verified without offering appropriate caveats is a problem with science reporting, not science.
The fact that the media is willing to trumpet results that haven't been independently verified without offering appropriate caveats is a problem with science reporting, not science.
39
Consider that there are undoubtedly many other papers that did not garner the same level of attention at this study did, but had the same shaky conclusions. An unfortunate side effect of a paper getting published, especially in a journal like Science, is that the conclusions are often canonized by the scientific community. This isn't to say that the truth will never come out; you're right, the scientific method is self-correcting. The problem is that the signal-to-noise ratio in scientific papers is not as high as it should be, and that is in part due to the harmful "publish or perish" mentality that exists in academia.
1
"The fact that the media is willing to trumpet results that haven't been independently verified without offering appropriate caveats is a problem with science reporting, not science."
The media has no subject matter expertise and defers to so-called experts. The onus is on the researchers to conduct their studies with integrity, not on the media to fact-check every claim.
The media has no subject matter expertise and defers to so-called experts. The onus is on the researchers to conduct their studies with integrity, not on the media to fact-check every claim.
1
Political and social "science" is not actually SCIENCE... It's not the same as evolution, astrophysics or mathematics. It's a "SOFT science" and subject to too much political and social interference. Such studies should always be taken with a grain of salt. Much like polling data
6
You are very right. The hard sciences have so much more rigor.
1
Reviewers can only review the distilled data presented to them in the manuscript at hand. Honesty is the coin of science. Deliberate lying is rare because, as in this case, others will attempt to reproduce the claims. If they can't, the fingers are quickly pointed at the investigators. The more sensationalist the claims, the quicker misconduct comes to light.
With papers published by graduate students the thesis supervisor commonly vouches for integrity. Though not made clear in the article, I suppose that must have been Dr. Vavreck at UCLA. In traditional doctoral theses, she would have worked most closely with Mr. LaCour evaluating the raw material of his study, and Dr. Green, as senior collaborator thousands of miles a way, should have been able to rely on her. By contrast, Mr. LaCour seemingly worked in a vacuum.
In addition, journals like Science require the disclosure of financial support with the submission of a research paper and print the information under the article, if published. Latest at submission flags should have been raised.
With papers published by graduate students the thesis supervisor commonly vouches for integrity. Though not made clear in the article, I suppose that must have been Dr. Vavreck at UCLA. In traditional doctoral theses, she would have worked most closely with Mr. LaCour evaluating the raw material of his study, and Dr. Green, as senior collaborator thousands of miles a way, should have been able to rely on her. By contrast, Mr. LaCour seemingly worked in a vacuum.
In addition, journals like Science require the disclosure of financial support with the submission of a research paper and print the information under the article, if published. Latest at submission flags should have been raised.
8
This article raises an important question about higher education today:
“They don’t care how well you taught. They don’t care about your peer reviews. They don’t care about your collegiality. They care about how many papers you publish in major journals.”
This raises concerns that education of students takes a back seat to "prestige" within academia.
“They don’t care how well you taught. They don’t care about your peer reviews. They don’t care about your collegiality. They care about how many papers you publish in major journals.”
This raises concerns that education of students takes a back seat to "prestige" within academia.
18
Anyone with a minimum of common sense and who was not living in a politically correct bubble, on reading about this "study" in the Times, knew immediately that it was self-serving -- or self-deceiving -- bunk.
4
it is also interesting that LaCour's adviser's name appears only in the final paragraph. Was she completely out of the loop in this study. I assume that is the case, but would that be business as usual in the department? She is a sometime contributor to the Times, so couldn't they ask her? This doesn't exonerate Green, though one doesn't expect such actions by a researcher, since the study was important regardless of the results, but maybe not to the mainstream folks; he wanted something that would grab the headlines, and proving what folks suspect is not exciting enough to get hired at Princeton.
7
We all tend to put way too much credence into "scientific" studies. Despite what they say most scientists have some type of bias going into an experiment and weeding out that bias is inherently difficult. In this case the lack of even basic scrutiny is appalling but not shocking given the fact that the outcome in this "experiment" was one the mainstream media would eagerly like to believe. I suspect the only way to trust this type of scientific research is to make sure that projects are run and managed by two or more people with opposite ideologies so that the results can withstand scrutiny.
5
I didn't pick this up in the first NY Times article on this issue, but it's clear to me now that Dr. Green was at a different university (Columbia) than LaCour and his adviser, Dr. Vavreck (UCLA). It appears that Dr. Green was acting more in an outside collaborator role than in an advisory role for the student's dissertation. It is odd to me that the student's actual adviser, Dr. Vavreck, is not listed as a co-author on the Science paper, but perhaps that's a common occurrence in social science than in my field of biomedical research.
The funding sources are clearly listed in the acknowledgements of the Science paper as the "Williams Institute, Ford Foundation, and Evelyn and Walter Haas, Jr., Fund". I'm impressed to find from LaCour's CV online that he's been able to raise over $790k in research grants to support his research, even before defending his PhD. $467k of that was to support the study in contention. I don't see how it's possible that, for that price tag, none of the work was performed. There is almost certainly more to the story here.
The funding sources are clearly listed in the acknowledgements of the Science paper as the "Williams Institute, Ford Foundation, and Evelyn and Walter Haas, Jr., Fund". I'm impressed to find from LaCour's CV online that he's been able to raise over $790k in research grants to support his research, even before defending his PhD. $467k of that was to support the study in contention. I don't see how it's possible that, for that price tag, none of the work was performed. There is almost certainly more to the story here.
The good news is that in trying to replicate the results of one potentially influential study, the study was found to be problematic. This is typical: it is very unlikely, practically impossible, for an influential study's results to be replicated over and over if the study is fraudulent. Only those studies that do not attract much interest, and hence nobody tries to replicate, are likely to survive as fraudulent studies, but then as they have no influence, who cares?
4
Who cares what the canvassers find! This is a civil rights issue. It's not about what you think. This is about consenting adults and their right to unite as they please with one another.
4
This has nothing to do with that issue. The point of the eXperiment is to see what happens when Y does N. You need to be honest in the results, period.
2
So let me see if i understand this correctly...One can conduct a study, as long as the results do not offend a politically correct, special interest group? If the study does offend such group, or have results contrary to what said group wants the public to believe, expect it to be challenged and denigrated. If it supports said group, all is well.
5
Ethics first, last and always!
Whatever the area of our society, and that includes some folks who tout themselves as religious, we seem to be failing when it comes to Ethics.
I weep for this country...
Whatever the area of our society, and that includes some folks who tout themselves as religious, we seem to be failing when it comes to Ethics.
I weep for this country...
8
The social sciences and humanities are in such a shambles today. Agenda-driven research has diminished its already debatable credibility. More often than not, the agenda and its foregone conclusions dictates the data gathering and analysis. At least this study was aided and abetted by senior faculty who should know better, right?
The irony isn't lost on me that there is little difference between a pseudo-research study conducted in the West and a earnest or narrowly self-serving and sloppy study done in the East these days. But try convincing smug academics in the West of that irony.
The irony isn't lost on me that there is little difference between a pseudo-research study conducted in the West and a earnest or narrowly self-serving and sloppy study done in the East these days. But try convincing smug academics in the West of that irony.
5
Completely agree that Dr. Green bears a lot of responsibility for not giving enough oversight to the project. In science, it is not wise to fake or fudge results. Either your discovery is so hot that other people will try to confirm it, and you will get caught. Or the discovery you publish is not very important, so it is not worth the risk to your scientific or personal reputation.
4
If I recall correctly, the Magazine published an article last year on a Dutch social scientist whose entire career seems to have been based on research that turned up provocative findings in sexy areas -- all based on fraud. In some cases there was no data at all. One of the things the article noted was that even when you seem to have a well-thought out study design and go to the effort of collecting and tabulating data in a rigorous way, you often find nothing significant. Which is not good if you are a social scientist who needs to publish, because you can't publish "no findings" over and over again. In thinking about this particular study, how exactly would you go about measuring the impact of canvassing on voting habits?
4
The shaking of trust is due to the fact that readers and even most practicing social scientists do not really know how commonly bogus data becomes the foundation for major research reports. Indeed, political science has been a leader among the social scientists in requiring the sharing of data sets along with the computer code needed to replicate published findings. In may fields such as psychology, data and questionnaires are considered proprietary, sharing of data is not part of the culture, and flawed research such as this could never be detected so swiftly -- if at all.
Still, it is clear that current efforts at data transparency do not go far enough. And the idea that Professor Greene was not permitted to examine the data because the study was not approved by his university's human subjects board identifies another weak link in the quality control nexus.
Still, it is clear that current efforts at data transparency do not go far enough. And the idea that Professor Greene was not permitted to examine the data because the study was not approved by his university's human subjects board identifies another weak link in the quality control nexus.
4
Anyone who has paid attention to much garbage pumped out of "social science" departments—especially during the past five decades—knows to take studies embracing social engineering with a large grain of salt.
As is usually the case, when the results are ideologically agreeable to the political left, they're heralded as "groundbreaking" and a "solution" to a "problem" as defined by the anointed—those in academia, the media, and elite press who want to remake the world according to their vision.
As is usually the case, when the results are ideologically agreeable to the political left, they're heralded as "groundbreaking" and a "solution" to a "problem" as defined by the anointed—those in academia, the media, and elite press who want to remake the world according to their vision.
3
This is one of the problems in the much-heralded era of "big data". Most of the scientists working with the data have not been involved in the collection of the data, and do not have intimate familiarity with the methods used. What to the data say about the frequency of disception in the social sciences compared with the biological sciences? This kind of cheating on the part of junior investigators is less feasible in the context of laboratory experiments that is closely supervised by senior investigators.
1
I guess there are two ways to make a name in the social sciences: one is to publish a landmark study, the other is to debunk the landmark study.
2
We live in an age of big data, or so we are told. When everything we do, say, think or even dream is monetized, then the currency - big data, dollars or number of published papers, assume a value on paper much larger than thought possible.
Recycling hope does not make one "greener." Greedier, yes.
Recycling hope does not make one "greener." Greedier, yes.
When many different people all want something badly enough – a big idea in science, a cure for cancer, a high appraised value of a home, free money, then checks and balances are removed and bad decisions are made. In science, reviewers are too busy to look into the details, they are too eager for the big idea to work, they have created a system that rewards novelty over all else, and new big ideas are rare. These ingredients underlie all fraud in science and in any other aspect of human society. The internet boom/bust, the housing bubble, fraud around stem cells and cold fusion fall into similar patterns. Film makers have a name for the key ingredient: suspension of disbelief.
There are two lessons:
1 – Human biology and psychology extends into every human endeavor and thus every human system is prone to inaccuracy and fraud.
2 – There will be fraud in science but it will eventually be detected and removed by the system. So over the long-term, science itself is to be trusted, even if not all scientists can be trusted all the time.
There are two lessons:
1 – Human biology and psychology extends into every human endeavor and thus every human system is prone to inaccuracy and fraud.
2 – There will be fraud in science but it will eventually be detected and removed by the system. So over the long-term, science itself is to be trusted, even if not all scientists can be trusted all the time.
2
The vetting aside, why is this such an important question? Other than for political canvassing - where to spend energy and money - does it really matter if a Gay person can influence someone's thinking? It seems pretty obvious to me that personal connections always influence to some degree. Did it need to be "studied?"
Personally, I am sick of surveys and studies with poorly worded questions with a gazillion possible nuances that draw "conclusions." I have been surveyed on the phone with such obviously slanted questions that I have asked the interviewer who is sponsoring the survey. One of them laughed and agreed with me that the question had no validity. Yet these are the things that we base decisions on? That run in the media (which is where the influence and shaping comes from)? To way too great an extent our lives are being unduly influenced by faulty polls and studies from everything from medical "information" to politics/candidates to education decisions.
Personally, I am sick of surveys and studies with poorly worded questions with a gazillion possible nuances that draw "conclusions." I have been surveyed on the phone with such obviously slanted questions that I have asked the interviewer who is sponsoring the survey. One of them laughed and agreed with me that the question had no validity. Yet these are the things that we base decisions on? That run in the media (which is where the influence and shaping comes from)? To way too great an extent our lives are being unduly influenced by faulty polls and studies from everything from medical "information" to politics/candidates to education decisions.
1
The system of peer review and replication, especially of surprising results basically worked here. The results could not be replicated, the irregularities, lies and lack of proper oversight were discovered and now have been revealed. Fabricating data is a cardinal sin in science. Mr. LaCour will have a very difficult time getting a job and will not be able to get grants. It is nearly impossible for journal editors or reviewers to detect completely fabricated data. As a scientist who regularly reviews manuscripts, I can tell you it is already enough work to do a thorough review. Expecting reviewers or editors to analyze raw data, as suggested by a few commenters, is completely impractical and authors who are willing to commit fraud could submit fabricated data sets anyway. Dr. Green certainly did not fulfill his obligation to properly supervise Mr. LaCour and his comments about the "sensitivity" of asking about seeing data or funding sources is nonsense. Most of the time you cite your funding source in the acknowledgements section of the paper because the sponsors want the (normally good) publicity. I went through at least parts of my raw data with every one of my supervisors on every study and every paper. It's just normal. Before condemning scientists and journals for the pitfalls of peer review, ask yourself which other discipline corrects itself as effectively.
Peer review is the worst form of evaluation, except for all the others that have been tried.
Peer review is the worst form of evaluation, except for all the others that have been tried.
16
So for half a century "scientists" have told us that all animals are monogamous, and anybody who isn't is an aberration. For half a century "scientists" have told us that gay people kill themselves, die early, and make lousy parents. But THIS -- a study that says face-to-face communication fights bigotry -- is the study that shakes our trust in science? Right.
5
What are you talking about? Which scientists say all animals are monogamous? And which scientists say gay parents are lousy? Both of those are contrary to the scientific consensus (the first is empirically false). You seem very confused!
This gets at a pet peeve of mine. I work in a scientific field, not social science, but one in which statistical analyses play a key role. I haven't never seen evidence of deliberate fraud, but what I've seen many times is a deliberate blind eye to verification of others' results. A lot of the work is based on fairly complex computer code to calculate -- or rather, to estimate -- various statistics + associated estimates of uncertainty, and there is NO verification, absolutely no validation whatsoever of the code or coding practices. It's 100% on the honor system. In the 15 years I've been in the field, I haven't heard a single person give lip service to the subject. (Okay, except for one fellow who gave a talk at a professional conference several years ago about medical research fraud he'd discovered, and how difficult it was to unearth and expose it.)
8
We need another word to describe wandering around or calling on the phone asking random people questions, expecting them to give honest answers, expecting those asking the questions to honestly report those answers, and then drawing some sort of conclusion from it. I don't know what it is, but it is not science.
8
I assume LaCour is gay. If so, it seems like he had a huge interest in the outcome. It's similar to those studies that concluded being gay is 100% nature and 0% nurture. They were funded by gay rights organizations. Such a finding is critical to much of the GLBT civil rights litigation.
Probably better to have unbiased people/entities conduct these sorts of studies.
Probably better to have unbiased people/entities conduct these sorts of studies.
5
Okay. Now explain how we could have avoided the Disney hogwash of scientists who proclaimed for the last fifty years that all animals are monogamous. Should we also discount "these sorts of studies" when they come from Christian heterosexuals?
In a brief paragraph about Princeton, this article also shows the flawed way prestigious universities pick their professors. Not on how well they teach but on which prestigious journals their papers have been published. The New York Times followed this study, which helped make the study prestigious as well.
7
Something tells me there was a definite hidden agenda behind this work: to further the notion that LGBT people somehow spread sexual preference which has strong evidence to have a genetic basis, fir examine based on evidence from non0human species. I wouldn't be surprised if Mr. LeCour's grants came from right wing evangelical Christian groups or right wing political entities.
3
Dig deeper and you'll find in academic departments wherein only advocates of feminism, for instance, will be hired for full time positions. Universities on the whole don't stand up for academic values unless they are under a spotlight--but look at the head of Harvard, a women with a CV below par for her area of supposed expertise. They likely won't stand up against, for instance, black students demanding that only a black professor fill a position in "Black Studies." And universities will create programs in about any area a political correct and stridently demanded.
7
If you are talking about Drew Gilpin Faust, I can assure you she is a well-respected historian of the South and the Reconstruction era.
5
Huh? You think Drew Gilpin Faust has a sub-par CV? I'd love to know what you think an average historican's CV looks like.
what's surprising is that people are surprised here. we know that a very high percentage of research, particularly in the social sciences, cannot be replicated. Some say its half. Sometimes the issue is fraud, as appears to be the case here. Sometimes it is a flawed plan that involves a minute sample. The media has an insatiable appetite for this stuff and routinely gives it far more credence than it deserves. It would be nice to get more reliable results. In the meantime, reporting that acknowledged that reported results were tentative and subject to change would be a positive step.
6
There is advocacy journalism and this is advocacy research. Reading the article, I notice a conspicuous absence of simple questioning and skepticism. I suspect that the politics of the study's researches, supervisors, and publishers line up with the study's results.
As I read the article, what popped into my mind immediately was the discredited Rolling Stone article on the University of Virginia.
As I read the article, what popped into my mind immediately was the discredited Rolling Stone article on the University of Virginia.
7
Speaking as a retired academic it would appear as if Mr. LaCour has failed in the most fundamental quality of a researcher, integrity. It would also appear as if Dr. Green has failed in the most fundamental quality of a senior scholar/mentor, careful and attentive supervision.
14
Co-authors who merely lend their well known name to a study, but are not participants in data collection and analysis, seem guilty of allowing the co-authorshipto be used to gain acceptance. One more paper for the everlasting requirement by which university teachers are judged.
My own limited experience as a referee indicates that it is who the author is and his relationships to editors that is more important and can over rule a negative report on a paper. - Dr. Green allowed himself to be used and was careless in not being a real co-author to an apparent fraud.
My own limited experience as a referee indicates that it is who the author is and his relationships to editors that is more important and can over rule a negative report on a paper. - Dr. Green allowed himself to be used and was careless in not being a real co-author to an apparent fraud.
3
No researcher who is personally invested in a specific outcome
of any experiment should be able to present findings supporting such a hypothesis. It is the antithesis of all scientific method.
if a man with a gun walked into a bank and began polling the customers, asking:"Don't you think , seeing as I have a gun pointed at your head, that this bank should give me all the money it has here?"
How many customers will disagree with the question?
When an interviewer is personally invested in a specific outcome of an interview, it is the same as arming the bank robber with a gun. People will naturally agree with him for reasons self preservation. Similarly, gay interviewers discussing and promoting opinions in favor of gay marriage, can convince themselves that they have been 100% persuasive and that , in reality, everyone "really agrees with me".
It is a desperately needy position, and many people will agree with them just to get away from them. This is not hard science and the Times should point this out--this is the mushiest-squishiest, softest of all social sciences and it is essentially mere opinion misrepresented as science .
of any experiment should be able to present findings supporting such a hypothesis. It is the antithesis of all scientific method.
if a man with a gun walked into a bank and began polling the customers, asking:"Don't you think , seeing as I have a gun pointed at your head, that this bank should give me all the money it has here?"
How many customers will disagree with the question?
When an interviewer is personally invested in a specific outcome of an interview, it is the same as arming the bank robber with a gun. People will naturally agree with him for reasons self preservation. Similarly, gay interviewers discussing and promoting opinions in favor of gay marriage, can convince themselves that they have been 100% persuasive and that , in reality, everyone "really agrees with me".
It is a desperately needy position, and many people will agree with them just to get away from them. This is not hard science and the Times should point this out--this is the mushiest-squishiest, softest of all social sciences and it is essentially mere opinion misrepresented as science .
1
As a 10-year veteran of canvassing door-to-door for political candidates, I would have taken with a big grain of salt any idea that someone with a particular viewpoint on an issue could change the position of someone by showing up on their doorstep or engaging in any other form of one-on-one interaction—much less, achieve significant results. It just does not happen. In fact, my first warning to a new group of canvassers is to put aside any self-regarding notion that they, personally, can sway opinions. They are more likely to accomplish the opposite: provoke shouting matches that will leave your interlocutors even more committed to their existing position. You are canvassing mainly to ensure that those already inclined to be in your camp know your candidate is running and go to the polls to vote. Believing that you have the oratorical skills of a Cicero or Lincoln can be dangerous.
This common sense knowledge would have greatly aided the academics that got sucked into this morass, quite aside from knowing about the skewed methods and data manipulation. This affair proves what I’ve known for some time: the more academically smart you are, the less effective you will be as a canvasser. You probably have an inflated view of your skills.
This common sense knowledge would have greatly aided the academics that got sucked into this morass, quite aside from knowing about the skewed methods and data manipulation. This affair proves what I’ve known for some time: the more academically smart you are, the less effective you will be as a canvasser. You probably have an inflated view of your skills.
1
I also canvass voters, and the received wisdom is that canvassing is the most important tool you can use to get people who already are inclined to vote your way to actually vote or register to vote. As you get closer to election day, that's why canvassing is targeted more and more at voters perceived to be already friendly. Not that it isn't worth studying, it is, but I would be skeptical that such a random, minimal amount of contact could have such a big impact.
As I like to say, the real review comes after the review. Scientists among the readers will know what I mean. For everyone else: The task of a peer reviewer is not to verify that work presented in a scientific manuscript is correct and true; for that, the peer reviewer would have to repeat the work done by the authors. Instead, a peer reviewer assesses a manuscript based on criteria such as impact, quality of precedures, internal consistency, consistency with the scientific method, presentation and writing, and embedding into existing scientific context. Once published, a manuscript is vetted by the scientific community. Good and bad apples are thereby identified.
Peer review is an activity performed by scientists who offer their time without pay or other rewards, time which they could use to do productive work. I recommend that critics of the current peer review process present better alternatives and means to fund them.
Peer review is an activity performed by scientists who offer their time without pay or other rewards, time which they could use to do productive work. I recommend that critics of the current peer review process present better alternatives and means to fund them.
3
There are several lessons here. First, the one the article focuses on: the pressure under which academics are to publish in respected journals that takes too many of them to fabricate data. And then there is the fact that several academics got their critical thinking clouded when it came to give a pass to a study that looked too good to be true only because they agreed with its conclusions. While in this case we have the extreme of fabricating data, academics supporting gay marriage have committed other excesses in the past. The two most egregious that come to my mind are : 1- to present studies that show that men married to women have better long term health outcomes than single men to support gay marriage for men, 2- studies with strong selection bias -namely by letting self selected upper middle class gay couples be the group to compare gay parenting against parenting by heterosexual couples- that pretend to show that gay parenting is as good as parenting by heterosexuals or in some cases better.
The public's trust in academia in the social sciences and humanities is at an all time low. That mistrust is also extending to other areas of science because of scandals like these. I also salute the NY Times courage for writing a story about this giving its own editorial line in favor of gay marriage.
The public's trust in academia in the social sciences and humanities is at an all time low. That mistrust is also extending to other areas of science because of scandals like these. I also salute the NY Times courage for writing a story about this giving its own editorial line in favor of gay marriage.
1
This was a nice piece of reporting on a riveting story. I'd only add a couple of points.
First, given that this article is in the Times' Science section, it's worth noting that the Kalla, Broockman and Aronow findings that cast doubt on the LaCour/Green results are much more detailed than the story suggests. The paper is at http://stanford.edu/~dbroock/broockman_kalla_aronow_lg_irregularities.pdf and I'd recommend it to anyone interested in the specifics.
Second, it seems that just about any mention of political science in the press brings out comments to the effect that political science (or social science more generally) isn't scientific. Granted, given that it's my job I'm not exactly unbiased. Even so, this case seems to me to prove precisely the opposite. The experimental design was awfully good—good enough for the editors of Science, which is pretty much the gold standard. It was clear enough to replicate. The replication failed, the original findings were called into question, and the result is what's written up here. That's actually how science works. Fraudulent studies happen in every discipline (remember cold fusion?), but only those disciplines that rely on the scientific method can ferret them out.
First, given that this article is in the Times' Science section, it's worth noting that the Kalla, Broockman and Aronow findings that cast doubt on the LaCour/Green results are much more detailed than the story suggests. The paper is at http://stanford.edu/~dbroock/broockman_kalla_aronow_lg_irregularities.pdf and I'd recommend it to anyone interested in the specifics.
Second, it seems that just about any mention of political science in the press brings out comments to the effect that political science (or social science more generally) isn't scientific. Granted, given that it's my job I'm not exactly unbiased. Even so, this case seems to me to prove precisely the opposite. The experimental design was awfully good—good enough for the editors of Science, which is pretty much the gold standard. It was clear enough to replicate. The replication failed, the original findings were called into question, and the result is what's written up here. That's actually how science works. Fraudulent studies happen in every discipline (remember cold fusion?), but only those disciplines that rely on the scientific method can ferret them out.
5
Bear's seems to be the best comment on this potentially misleading study. There is indeed plenty of culpability here, the advisor, Dr. Green, Princeton University. However, it could be argued that this is an example of how science should work and that changes in public policy are not needed. Published studies have to be described precisely enough that they can be replicated. Replication by an unaffiliated group is the ultimate test of believability, and many working scientists will not believe an important finding until it has been replicated. All the details about pressure on young scientists, inability of reviewers to fully evaluate conclusions, deliberate or unconscious bias, etc. are always concerns with every published result. The best long-term solution is to encourage publication in high profile journals of replications and systematic extension of previous findings. Today, novelty is so prized that the first group reporting something interesting gets all the credit, while replications are considered much less news worthy and many authors will not pursue publication of replications once scooped. Working scientists are almost always aware of ongoing replications and adjust their certainty about findings accordingly. On the other hand, those not close to an area may feel they are getting jerked around by inconsistent results. When some conclusion is politically important, a solution is to ask all the scientists working in a field (e.g. global warming) for their view..
Correct me, if I am wrong, 'the study' examined canvassers with a personal stake in an issue - do they sway voters’ opinions in a lasting way?
The way this article is written, I doubt many readers will be able to follow the reported events and people. The headline 'Study on Gay Unions' guarantees it.
I looked to the top of the column to see if Mr. LaCour's name was on this 'report' too. You have my permission to retain, verify and report my 'data.'
The way this article is written, I doubt many readers will be able to follow the reported events and people. The headline 'Study on Gay Unions' guarantees it.
I looked to the top of the column to see if Mr. LaCour's name was on this 'report' too. You have my permission to retain, verify and report my 'data.'
1
Perhaps, just perhaps, the most alarming facet of the story is that the editors at Science were still "grappling with decision on retracting the study." And yes, we should all just trust Hillary and her advisors' vetting of her emails......
Professor Green was willfully blind to the problems with the raw data, blinded by the possibility of getting published in Science, blinded by the charms of his graduate student.
Science is all about the data, if the data is flawed or fraudulent or missing or erased (lol), you have nothing but conjecture and hypothesis. No leading scientific journal would have published Professor Green's conjecture and hypothesis alone. The missing data is Professor Green's responsibility.
Science is all about the data, if the data is flawed or fraudulent or missing or erased (lol), you have nothing but conjecture and hypothesis. No leading scientific journal would have published Professor Green's conjecture and hypothesis alone. The missing data is Professor Green's responsibility.
3
I am a scientist and have watched as 'science' has been abducted by politics. Politics have always had an influence as the vast majority of grant monies come from government. If we believe that the CDC or NIH (or any of the 'scientifice' government agencies OR private funding groups) are not polical then we are quite naive.
For example, if you want a grant from the government you will likely not get it unless climate change is included. That's true even when your study has nothing to do with the weather - e.g. menstration patterns in rats as a result of applied low dose radiation (or as a result of increased vitamin C in the diet). I'm not saying anything about climate change, whether it is the result of increased CO2 or any other human driven activity. I'm saying that we have an extreme political influence in science, always have, and that climate change is one of the most onerous impacts on grant monies over the last 20 years. Government has an interest in this as it justifies Cap and Trade and empowers a bunch of agenda driven UN members. That is one of the reasons many have doubts as to the extent of impact, especially when media is so ready to jump on every graph or conclusion.
Shame on any scientist that does not have an open mind. Shame on any politician or activist investor that believes/trys to influence the outcome of a study. But since this has gone on forever, shame on the public that is too naive/lazy to understand that one study isn't proof.
For example, if you want a grant from the government you will likely not get it unless climate change is included. That's true even when your study has nothing to do with the weather - e.g. menstration patterns in rats as a result of applied low dose radiation (or as a result of increased vitamin C in the diet). I'm not saying anything about climate change, whether it is the result of increased CO2 or any other human driven activity. I'm saying that we have an extreme political influence in science, always have, and that climate change is one of the most onerous impacts on grant monies over the last 20 years. Government has an interest in this as it justifies Cap and Trade and empowers a bunch of agenda driven UN members. That is one of the reasons many have doubts as to the extent of impact, especially when media is so ready to jump on every graph or conclusion.
Shame on any scientist that does not have an open mind. Shame on any politician or activist investor that believes/trys to influence the outcome of a study. But since this has gone on forever, shame on the public that is too naive/lazy to understand that one study isn't proof.
5
Rats don't menstruate
The verdict on this case is not yet in, and all the facts are not yet in the public eye. But from what appears in this article, it seems to be a case of outright fraud and data fabrication, one of the most onerous and damaging forms of scientific misconduct.
If the circumstances prove to be as the article suggests, there are two guilty parties: Mr. LaCour, who apparently committed the fraud, and Dr. Green, his long distance supervisor and mentor, who lent his name to the work, thereby giving the research the credibility it likely needed to gain publication in the prestigious journal Science. It would seem Dr. Green, who lived and worked a continent away, appears not to have provided even rudimentary review of the project he allowed to appear under his name. It’s hard to adequately supervise when you’re not there.
For several similar examples, from the “hard” biological sciences, which occurred in an earlier era, readers can google the names Robert Slutsky or John Darsee.
One can’t really fault the journal Science, and I certain don’t think Dr. McNutt should be made to fall on her sword. Good editing and peer review (which as the Times notes is voluntary and unpaid) are not equipped to detect frank dishonestly. Requiring that raw data be published will help, but given the requirement to protect the confidentially of research subjects, even this this will not catch all cases.
Fortunately, outright fabrication of data truly is rare.
If the circumstances prove to be as the article suggests, there are two guilty parties: Mr. LaCour, who apparently committed the fraud, and Dr. Green, his long distance supervisor and mentor, who lent his name to the work, thereby giving the research the credibility it likely needed to gain publication in the prestigious journal Science. It would seem Dr. Green, who lived and worked a continent away, appears not to have provided even rudimentary review of the project he allowed to appear under his name. It’s hard to adequately supervise when you’re not there.
For several similar examples, from the “hard” biological sciences, which occurred in an earlier era, readers can google the names Robert Slutsky or John Darsee.
One can’t really fault the journal Science, and I certain don’t think Dr. McNutt should be made to fall on her sword. Good editing and peer review (which as the Times notes is voluntary and unpaid) are not equipped to detect frank dishonestly. Requiring that raw data be published will help, but given the requirement to protect the confidentially of research subjects, even this this will not catch all cases.
Fortunately, outright fabrication of data truly is rare.
3
Just imagine if this were occurring in a field where almost all the results have political implications and where the field were completely dominated by a particular political viewpoint...
4
We'd find the same result. All research gets challenged at some point, sometimes before it's even published. For example, before studies were published so many decades ago about the cancer-causing effects of smoking, Big Tobacco issued its own campaign to cast doubt on the research and keep people "safely" lighting up by promoting "filtered" cigarettes. Many millions of lives and billions of dollars were lost in that public debate of the Truth.
Shouldn't this event shake confidence? Brazen evidence falsification backing research submitted to Science should make us question most research that never attracts sponsorship taking it so far.
The giveaway is researchers saying "I stand by the results." with no elaboration or explanation. That statement's users realize that they erred OR as here that they have been caught fudging the science. Seeking to put a brave face on the affair, they posture. If they really stand by the results, they cannot object to explaining how they got those results.
The latest near example springing to mind is Dan Rather. In that case, he eagerly swallowed a Bush-Jr story whose anachronisms did not stack up then reported it as fact.
The giveaway is researchers saying "I stand by the results." with no elaboration or explanation. That statement's users realize that they erred OR as here that they have been caught fudging the science. Seeking to put a brave face on the affair, they posture. If they really stand by the results, they cannot object to explaining how they got those results.
The latest near example springing to mind is Dan Rather. In that case, he eagerly swallowed a Bush-Jr story whose anachronisms did not stack up then reported it as fact.
1
This article doesn't offer what those "statistical manipulations" might have been. Perhaps the magazine, Science, will. Statistics can back up that you are studying what you think you are studying. The data, on the other hand, can point to more than one variable in play, or that another variable(s) is worthy of study before drawing conclusions. In this study, "feelings thermometer" sounds very quasi in the first place; someone needs to explain this.
In a recent Yale study, the reason given, from questioning, why those turning 18 in Conn. did not vote, was that they were unsure the voting process was secret. A better explanation (my opinion) was they were apathetic, and used the secrecy of the ballot as an excuse, rather than admit they were apathetic.* Or, in a "feelings" survey, how do you know the respondents are not lying, or are not influenced by the way the questions are stacked, or in this instance (gay issues), answered the questions to please the interviewer, just to make him/her go away.
* What this study (Gerber et al) showed was that young people, that had recently been walked through the voting process as part of their education in high school, were much more likely to vote than those who hadn't.
In a recent Yale study, the reason given, from questioning, why those turning 18 in Conn. did not vote, was that they were unsure the voting process was secret. A better explanation (my opinion) was they were apathetic, and used the secrecy of the ballot as an excuse, rather than admit they were apathetic.* Or, in a "feelings" survey, how do you know the respondents are not lying, or are not influenced by the way the questions are stacked, or in this instance (gay issues), answered the questions to please the interviewer, just to make him/her go away.
* What this study (Gerber et al) showed was that young people, that had recently been walked through the voting process as part of their education in high school, were much more likely to vote than those who hadn't.
2
The is no such thing as a "gay union" or "gay marriage."
The very use of the terms promotes discrimination, which is not based on any facts, but which is instead purely ideological.
The correct terms are "marriage equality" and "equal protection under the law."
The very use of the terms promotes discrimination, which is not based on any facts, but which is instead purely ideological.
The correct terms are "marriage equality" and "equal protection under the law."
3
This article is far too lenient toward Dr. Green, who has consistently refused to take any responsibility for his actions, or lack thereof in the matter, even though he took credit for the study before it was discredited.
Science has explicit standards for co-authorship, and the responsibilities of "senior" authors, specifically "The senior author from each group is required to have examined the raw data their group has produced." is here: http://www.sciencemag.org/site/feature/contribinfo/prep/gen_info.xhtml#a... . Basically, Dr. Green, as senior author, signed a document with Science stating that he personally, reviewed the raw data and attested to its validity. He apparently lied to Science, and he continues to lie to the media.
And shame on the New York Times for not acknowledging that Lynn Vavreck is not just Mr. LaCour's academic advisor at UCLA, she is a full professor (not an associate professor as reported), she is Mr. LaCour's doctoral dissertation chair, and more importantly, she is a New York Times contributor - a clear conflict of interest: http://www.nytimes.com/2015/02/19/upshot/why-network-news-still-matters....
The latter ought to be corrected and reported to the NYT Ombudsperson.
Science has explicit standards for co-authorship, and the responsibilities of "senior" authors, specifically "The senior author from each group is required to have examined the raw data their group has produced." is here: http://www.sciencemag.org/site/feature/contribinfo/prep/gen_info.xhtml#a... . Basically, Dr. Green, as senior author, signed a document with Science stating that he personally, reviewed the raw data and attested to its validity. He apparently lied to Science, and he continues to lie to the media.
And shame on the New York Times for not acknowledging that Lynn Vavreck is not just Mr. LaCour's academic advisor at UCLA, she is a full professor (not an associate professor as reported), she is Mr. LaCour's doctoral dissertation chair, and more importantly, she is a New York Times contributor - a clear conflict of interest: http://www.nytimes.com/2015/02/19/upshot/why-network-news-still-matters....
The latter ought to be corrected and reported to the NYT Ombudsperson.
24
Not only did the study itself 'fail on all counts', the media helped over-report the study and is now under-reporting its share of responsibility for this debacle.
3
Mainstream "science," especially the so-called social sciences, was long ago subordinated to the liberal elite political agenda. Historically, the consent of the masses to any particular government policy depended on that government policy satisfying the public's "common sense" as based upon eons of empirical evidence. The masses were willing, at times, to accept counter-intuitive government policies which were based on hard science.
The Brandeis "social brief" changed all that. The use and over-use of dubious "studies," passed off as "science," in determining judicial and political issues has ultimately eroded acceptance by large portions of the masses of any "science" as a rationale for abandoning common sense, traditional solutions to cultural and political issues.
The Brandeis "social brief" changed all that. The use and over-use of dubious "studies," passed off as "science," in determining judicial and political issues has ultimately eroded acceptance by large portions of the masses of any "science" as a rationale for abandoning common sense, traditional solutions to cultural and political issues.
Seems like the universities need to get away from publication in major journals as the only way to judge merit in a professor, kind of obviously problematic on several levels. If you dug into their reasoning a bit I'm sure you would find the same substance that contaminates just about every other area of human endeavor, money.
1
Really? You mean respectful canvassers can change a person's view point on a controversial subject? So measuring this statistically is worth publishing? I think it boils down to money- the need for one to publish, the need for a journal to grab attention. How sad and ridiculous.
Science is not flawed, but it is practiced by humans. By virtue of our shared biology, all humans, including scientists and non-scientists, are imbued with qualities that make us prone to the deceptive practices of persuasive hucksters. However, science must be repeated, and all important fabrications are eventually revealed. High profile incidents of deception might lead one to question the integrity of the whole process, but in fact the process is like a self-righting boat: a strong wave from the right direction at the right time might capsize it, but it will right itself eventually.
As someone who has been in science many years, I have seen a clear pattern to these cases: a single bad actor, usually a trainee, with strong powers of persuasion and a big idea can convince otherwise skeptical, rational name brand scientists to invest in the big idea without the usual levels of proof they might require. Involvement of the name-brand scientist allows the idea to sail through subsequent levels of skeptics unchallenged. It is usually scientists down in the trenches who fail to repeat the result that uncover its fraudulent nature. The result is a great explosion that damages people, institutions, journals, and everyone within the blast zone.
As someone who has been in science many years, I have seen a clear pattern to these cases: a single bad actor, usually a trainee, with strong powers of persuasion and a big idea can convince otherwise skeptical, rational name brand scientists to invest in the big idea without the usual levels of proof they might require. Involvement of the name-brand scientist allows the idea to sail through subsequent levels of skeptics unchallenged. It is usually scientists down in the trenches who fail to repeat the result that uncover its fraudulent nature. The result is a great explosion that damages people, institutions, journals, and everyone within the blast zone.
6
Both my wife and I are physical scientists. Neither of us has ever had a co-author who simply "signed-off" on a journal submission. That was especially true when we were grad students and remains so to this day. On more than one occasion a co-author has insisted on a complete re-write of sections of a paper before submission.
Such scrutiny must begin with the basic data and methodology of the project. If not, then GIGO, Garbage In, Garbage Out.
Such scrutiny must begin with the basic data and methodology of the project. If not, then GIGO, Garbage In, Garbage Out.
7
Any study which goes against the modern cultural bent towards advocacy of same sex marriage rights, and the leftward journalistic zeal to defend the same, would inevitably face withering criticism and a PR campaign to debunk its findings. It is a mark of the times that any study that dares confront the gay lobby's orthodoxy would face this sort of fearful scrutiny. For to accept the possible legitimacy of any findings that threaten the trending groupthink on this issue, is to admit that public debate on the topic of same sex marriage is in order. That of-course is unthinkable to same sex marriage proponents, therefore any study that goes against the apriori worldviews of such proponents must of necessity be illegitimate.
5
? That's kind of not the point being made here. Specifically, whether a gay marriage opponent is persuaded more by gay canvasser (those with an opposite self-interest from those being canvassed) or not, how does this fit into your comment? Yes, people from BOTH sides try to pick apart studies whose results they don't like, but a good study withstands scrutiny, and the flaws in this study were pointed out by people trying to replicate such results in their own study.
The fact that this is even an issue gives dismaying credibility to the mistaken notion that human equality and civil rights are social experiments determined by popular opinion. Studies show that studies can show pretty much whatever you want them to show.
15
You need a qualifier in that last sentence.
Fraudulent or botched studies can show pretty much whatever you want them to show. Properly conducted studies that have a large enough sample size and are properly analyzed are fairly reliable and can be trusted. (Modulo the usual frequentist caveat that the methods are constructed so as to result in Type I error [rejecting the null hypothesis when it is, in fact, true] just 5% of the time.)
Fraudulent or botched studies can show pretty much whatever you want them to show. Properly conducted studies that have a large enough sample size and are properly analyzed are fairly reliable and can be trusted. (Modulo the usual frequentist caveat that the methods are constructed so as to result in Type I error [rejecting the null hypothesis when it is, in fact, true] just 5% of the time.)
I see several kinds of insidious fraudulence in social- and political-scientific practice, including the kind reported here.
Surveys lousy with poorly-worded and simplistic questions are conducted and then taken to support this or that grand thesis about well-being, happiness, worldview, etc. Something unspeakably complicated is reduced to a crisply articulated claim to which, it is thought, participants or subjects would of course assent. Meanings of certain crucial bits of vocabulary are presumed to be intersubjectively settled. A connection is imagined between the way one answers a survey and the way one thinks about a topic -- as if worldviews developed through lifetimes of experiences can be captured on a spreadsheet.
What do you suppose it is to "track voters' attitudes toward same-sex marriage?" Are there a handful of surveyable attitudes toward same-sex marriage? What does it take for a data set to represent an attitude? Does a change in vote entail a change in attitude? What about vice versa? Why would an attitude ever be representable as a series of answers to yes-no questions?
These kinds of fraudulence are just as serious, if not more so, than the kind reported here. But they are certainly not as eye-catching and newsworthy.
Surveys lousy with poorly-worded and simplistic questions are conducted and then taken to support this or that grand thesis about well-being, happiness, worldview, etc. Something unspeakably complicated is reduced to a crisply articulated claim to which, it is thought, participants or subjects would of course assent. Meanings of certain crucial bits of vocabulary are presumed to be intersubjectively settled. A connection is imagined between the way one answers a survey and the way one thinks about a topic -- as if worldviews developed through lifetimes of experiences can be captured on a spreadsheet.
What do you suppose it is to "track voters' attitudes toward same-sex marriage?" Are there a handful of surveyable attitudes toward same-sex marriage? What does it take for a data set to represent an attitude? Does a change in vote entail a change in attitude? What about vice versa? Why would an attitude ever be representable as a series of answers to yes-no questions?
These kinds of fraudulence are just as serious, if not more so, than the kind reported here. But they are certainly not as eye-catching and newsworthy.
15
Yes, I agree completely. The big problem lurking behind this issue is the belief that "data" actually gives us insight into human beings and that, as you note, complicated issues and processes and beliefs, etc. can be reduced to quantifiable, operationalizable concepts suitable for "scientific" study.
The article gets several facts wrong, and as a result completely misunderstands the story.
Everyone, including Green, had access to the complete dataset and materials – every question, every respondent’s response, and every bit of code used to produce the results. What they did not have access to was the RAW dataset as downloaded from the online surveyer (with social security numbers and other private information, scattered through numerous files). For Green recheck all these files would be incredibly time-consuming, not to mention a violation of subject rights.
The CLEANED complete dataset is what Broockman and Kalla used to investigate the statistical anomalies that spurred the retraction. Response rate was just the initial issue that led them to more clear-cut signs of fraud - mostly having to do with the distribution of one of the dependent variables.
Green in fact did everything that was expected of him. LaCour came to him with an already-assembled, cleaned dataset. Green examined the data, helped design the statistical analysis, and replicated all results. Because academic departments monitor incoming grants, he took LaCour at his word about his grant.
This story is not about lack of oversight – it’s about how one person violated every tenet of academic integrity and was discovered because of the growing trend of data transparency.
Read the actual files: http://stanford.edu/~dbroock/broockman_kalla_aronow_lg_irregularities.pdf
Everyone, including Green, had access to the complete dataset and materials – every question, every respondent’s response, and every bit of code used to produce the results. What they did not have access to was the RAW dataset as downloaded from the online surveyer (with social security numbers and other private information, scattered through numerous files). For Green recheck all these files would be incredibly time-consuming, not to mention a violation of subject rights.
The CLEANED complete dataset is what Broockman and Kalla used to investigate the statistical anomalies that spurred the retraction. Response rate was just the initial issue that led them to more clear-cut signs of fraud - mostly having to do with the distribution of one of the dependent variables.
Green in fact did everything that was expected of him. LaCour came to him with an already-assembled, cleaned dataset. Green examined the data, helped design the statistical analysis, and replicated all results. Because academic departments monitor incoming grants, he took LaCour at his word about his grant.
This story is not about lack of oversight – it’s about how one person violated every tenet of academic integrity and was discovered because of the growing trend of data transparency.
Read the actual files: http://stanford.edu/~dbroock/broockman_kalla_aronow_lg_irregularities.pdf
114
Respectfully,
You say that Dr Green didn't ask about funding sources because "academic departments monitor incoming grants." How do you know that was his reason? Because Dr Green is quoted giving an entirely different reason: “But it’s a delicate matter to ask another scholar the exact method through which they’re paying for their work.”
You say that "LaCour came to him with an already-assembled, cleaned dataset." But from the article, part of the data was collected after Dr. Green started becoming involved, when "Dr. Green insisted the work be replicated."
You say that it would have been "a violation of subject rights" for Dr Green, who is a co-author of the research study, to look at the raw data on which that study is based. How does that make sense? A co-author of a research paper has the right to look at raw data.
You say that Dr Green didn't ask about funding sources because "academic departments monitor incoming grants." How do you know that was his reason? Because Dr Green is quoted giving an entirely different reason: “But it’s a delicate matter to ask another scholar the exact method through which they’re paying for their work.”
You say that "LaCour came to him with an already-assembled, cleaned dataset." But from the article, part of the data was collected after Dr. Green started becoming involved, when "Dr. Green insisted the work be replicated."
You say that it would have been "a violation of subject rights" for Dr Green, who is a co-author of the research study, to look at the raw data on which that study is based. How does that make sense? A co-author of a research paper has the right to look at raw data.
3
I have trouble accepting your defense of Green. Looking at the raw data would be unethical? I don't think so --- not for the senior author on the study. It seems to me that he had a DUTY to examine that data before checking that box on the manuscript submission form stating that he swore to the accuracy of the paper's contents. And the fact that LaCour wasn't posting the data to that secure site, as Green himself asked him to do, should have raised every red flag Green had.
As for not asking about LaCour's funding source? Give me a break. Journals REQUIRE that disclosure, so it's not exactly a secret to be guarded (and is a problem if it is). Moreover, any junior researcher with the kind of funding LaCour was claiming to have would have been volunteering the information to impress a senior researcher with whom he wanted to collaborate. And asking about someone's funding source isn't "delicate." It's SOP, and even a normal part of many casual conversations about one's research.
While LaCour's actions were obviously wrong, Green isn't above reproach, either. As a senior researcher, he should have known better. It's his job to know better.
As for not asking about LaCour's funding source? Give me a break. Journals REQUIRE that disclosure, so it's not exactly a secret to be guarded (and is a problem if it is). Moreover, any junior researcher with the kind of funding LaCour was claiming to have would have been volunteering the information to impress a senior researcher with whom he wanted to collaborate. And asking about someone's funding source isn't "delicate." It's SOP, and even a normal part of many casual conversations about one's research.
While LaCour's actions were obviously wrong, Green isn't above reproach, either. As a senior researcher, he should have known better. It's his job to know better.
3
I have trouble accepting your defense of Green. Looking at the raw data would be unethical? I don't think so --- not for the senior author on the study. It seems to me that he had a DUTY to examine that data before checking that box on the manuscript submission form stating that he swore to the accuracy of the paper's contents. And the fact that LaCour wasn't posting the data to that secure site, as Green himself asked him to do, should have raised every red flag Green had.
As for not asking about LaCour's funding source? Give me a break. Journals REQUIRE that disclosure, so it's not exactly a secret to be guarded (and is a problem if it is). Moreover, any junior researcher with the kind of funding LaCour was claiming to have would have been volunteering the information to impress a senior researcher with whom he wanted to collaborate. And asking about someone's funding source isn't "delicate." It's SOP, and even a normal part of many casual conversations about one's research.
While LaCour's actions were obviously wrong, Green isn't above reproach, either. As a senior researcher, he should have known better. It's his job to know better.
As for not asking about LaCour's funding source? Give me a break. Journals REQUIRE that disclosure, so it's not exactly a secret to be guarded (and is a problem if it is). Moreover, any junior researcher with the kind of funding LaCour was claiming to have would have been volunteering the information to impress a senior researcher with whom he wanted to collaborate. And asking about someone's funding source isn't "delicate." It's SOP, and even a normal part of many casual conversations about one's research.
While LaCour's actions were obviously wrong, Green isn't above reproach, either. As a senior researcher, he should have known better. It's his job to know better.
1
This clear, unambiguous statement tells us that a specific finding is false in the face of accusations of misrepresentation: "...[A] finding that gay canvassers were in fact powerfully persuasive with people who had voted against same-sex marriage... collapsed amid accusations that Mr. LaCour had misrepresented his study methods and lacked the evidence to back up his findings."
How is it possible that the findings collapsed if the study data sets are not available to test for that conclusion? Collapsing findings based on speculation that the data for the data does not exist to support the study findings uses, as useful to its finding, the very thing it condemns: speculation as to the existence of data.
How is it possible that the findings collapsed if the study data sets are not available to test for that conclusion? Collapsing findings based on speculation that the data for the data does not exist to support the study findings uses, as useful to its finding, the very thing it condemns: speculation as to the existence of data.
5
Some of Dr Green's statements don't seem to make any sense.
-- He's afraid to ask a co-author / mentee for a look at his raw data? Ridiculous on its face.
-- “But it’s a delicate matter to ask another scholar the exact method through which they’re paying for their work.” huh? -- scholarly work typically includes acknowledgement of funding sources -- in the interests of full disclosure.
-- He's afraid to ask a co-author / mentee for a look at his raw data? Ridiculous on its face.
-- “But it’s a delicate matter to ask another scholar the exact method through which they’re paying for their work.” huh? -- scholarly work typically includes acknowledgement of funding sources -- in the interests of full disclosure.
13
This whole article was puzzling from the headline to the photograph (which may give some the impression that they are a gay couple) to the writing within. That being said, I don't fault the Times. They wrote it and I chose to read it to escape my own problems this morning.
I agree that Dr. Green's statements are strange. Who's the boss here?
I agree that Dr. Green's statements are strange. Who's the boss here?
7
This whole matter seems so strange to me. I am also a PhD and I have written scholarly journal articles. Believe me, if my name is going on an article, I *am* asking lots of questions.
1
In academic publications with multiple authors, the senior author often is completely detached from the actual study or publication. It's all about padding one's resume.
5
Point taken, but on the other hand, the senior co-author has a reputation to guard.
You are absolutely correct; I was just pointing out the reality that exists in spite of that interest to protect one's reputation. I am always amazed when a senior author cannot even begin to recite the basic background, research and conclusions, or responsibilities of the junior authors, of the studies beyond what is published.
My calculations suggest that the studies would have cost a minimum of $750,000 to $800,000 to run as reported. There is no way that a GRADUATE student walks around with that kind of funding and no one knows about it or where it comes from. It is simply NOT a "delicate" matter to ask. I have no idea what universe these folks live in.
50
It is utterly disingenuous to suggest, as this article does, that Professor Green was nothing more than a kind senior scholar out to help out a graduate student, who did not want to overstep his bounds and thus did not insist on looking at the raw data.
No, let's be clear: Professor Green chose to be given credit as co-author of an article that appeared in Science. With co-authorship in Science comes not only the prestige and recognition, but ownership, responsibility and obligations.
1) As Prof. Green admits, the article was too ambitious for a graduate student alone, thus it needed the imprimatur of a senior scholar such as himself. Science probably would not have published the graduate student's data alone, it needed the Prof. Green's stamp of approval. Other scholars who might have doubted the work found assurance in Prof. Green's co-authorship.
2) Co-authorship in Science is not a trivial matter, a lot of prestige comes with it, prestige that Prof. Green was all too happy to enjoy. But with the prestige comes the obligations of vetting all the data, vouching for the data, or at minimum, having looked at the raw data. Prof. Green had never even seen the raw data.
Prof. Green got what he wanted, 50% credit for the article, now he gets 50% blame for the faults as well. He can't just disavow the article as an "oops" or an "embarrassment", not of his making, for in signing on to the article, he signaled to the world he examined all the data, including the raw data.
No, let's be clear: Professor Green chose to be given credit as co-author of an article that appeared in Science. With co-authorship in Science comes not only the prestige and recognition, but ownership, responsibility and obligations.
1) As Prof. Green admits, the article was too ambitious for a graduate student alone, thus it needed the imprimatur of a senior scholar such as himself. Science probably would not have published the graduate student's data alone, it needed the Prof. Green's stamp of approval. Other scholars who might have doubted the work found assurance in Prof. Green's co-authorship.
2) Co-authorship in Science is not a trivial matter, a lot of prestige comes with it, prestige that Prof. Green was all too happy to enjoy. But with the prestige comes the obligations of vetting all the data, vouching for the data, or at minimum, having looked at the raw data. Prof. Green had never even seen the raw data.
Prof. Green got what he wanted, 50% credit for the article, now he gets 50% blame for the faults as well. He can't just disavow the article as an "oops" or an "embarrassment", not of his making, for in signing on to the article, he signaled to the world he examined all the data, including the raw data.
76
As a retired psychology professor who taught survey research methods and previously worked at The University of Michigan's prestigious Institute for Social Research, I'm shocked at the stunning lack of oversight by Professor Donald Green. Survey research requires a highly-trained field staff as well as carefully designed survey questions. It was Professor Green's duty to be involved in both these aspects and perhaps the development of the sampling frame (the list used to randomly select respondents to be surveyed). Science requires such sound research methodology in order to produce valid results that can pass the scrutiny of replication and reanalysis. It seems this study failed on all counts.
95
I agree, Paul. I read this article, and concluded that either things have really changed since I was a graduate student in the biological sciences in the late 70s, or that the "social sciences" are not really science at all. I can't imagine a research supervisor making the inane statements that Dr. Green is. Talk about abdicating responsibility, then furiously backpedaling to justify his bizarre behavior. Atrocious. And he's a professor at Columbia University, no less. I have much, much less respect for that institution now, which apparently employs such inept incompetent (and possibly worse) faculty.
1
Perhaps, just perhaps, the study was funded by the Koch brothers in order to establish a legal basis for
excluding all testimony of scientific experts in courts of law in USA. Now wouldn't that be an interesting
Supreme Court dexision, if it were to happen. What a delicious conspiracy theory!
excluding all testimony of scientific experts in courts of law in USA. Now wouldn't that be an interesting
Supreme Court dexision, if it were to happen. What a delicious conspiracy theory!
Research dishonesty does not start in graduate school but might be endemic throughout the entire educational system. I think that it starts with a lackadaisical attitude of the instructor toward things like plagiarism on essays and research papers. A case in point is when I was teaching a course on urban sociology at an undergraduate college in New York city and one of the students handed in a paper that was entirely copied from a magazine article. After talking to the student and explaining the possible ramifications we were walking down the hallway and ran into another professor of his. The student told the professor what had happened and said to the professor, ":I do that in all my essays and you have always given me an A." In short, dishonesty, whether to get a prestigious job or not, is taught. It is not natural.
It is incumbent to find ways to reverse this learning trend.
It is incumbent to find ways to reverse this learning trend.
19
Poor choice of headline, NYT. This was not a study on gay unions, it was a study on voter persuasion.
108
It would seem reasonable to suggest that when the results of a study gratify a substantial portion of the population the study should be under extra scrutiny.
6
Nobody should be allowed to publish without releasing the original raw data. Nobody would be allowed to publish an organic chemistry article on producing a new compound without printing how they did it. "Trust me, it's all legit... now lets look at the results" just doesn't cut it in most fields. Allowing others to independently reproduce results is a core rule in many/most fields and yet some of the softer sciences like economics and social sciences seem to try and skirt this rule.
If you're not willing to let others look at the same dataset and test your analysis and conclusions then it's probably not a piece of work worth publishing--and a prestigious journal like Science should have known better than to print something without releasing, or even reviewing, the original data.
If you're not willing to let others look at the same dataset and test your analysis and conclusions then it's probably not a piece of work worth publishing--and a prestigious journal like Science should have known better than to print something without releasing, or even reviewing, the original data.
13
In high school, long before I became a real scientist, I messed up a chemistry lab and had to make up fake data for my lab report. It was immediately apparent to me that I would get caught if my numbers were too clean. But this is exactly what trips up professional frauds, from Cryil Burt in psychology to Hendrik Schön in physics, and now this clown: their data were too good, impossibly reproducible, easy to detect once you start looking closely. Our scientific/medical complex has a serious problem: our placebos are not getting any more effective, and our frauds are not getting any smarter.
In my own case, I swear before the honorable readers of the New York Times that I myself am like the priest in the old joke who admits to his friend that rabbi that he is not actually a virgin: I only did it once, long before I was ordained.
In my own case, I swear before the honorable readers of the New York Times that I myself am like the priest in the old joke who admits to his friend that rabbi that he is not actually a virgin: I only did it once, long before I was ordained.
8
Ego, greed and agenda have so permeated the academic community that even a once unimpeachable journal like Science isn't immune. It's truly distressing to see Science emerge as a tool of those with a political/social goal rather than true scientific curiosity.
11
Contrary to the headline, this was not a 'maligned' study. It was a fraudulent 'study'.
43
I think that the problem here is with Dr. Green.
I have it on good authority that publication in a name journal requires a "name" co author. So Mr. LaCour enlisted Dr. Green who lent his name to the study with little or no oversight on his part.
One of the thing I have learned in life is that when I sign something I take responsibility for it. So when Dr. Green signed the paper, he took co-responsibility for it's contents. But he utterly failed to make any attempt to check the correctness of the application of the data to the results, and therefore permitted a questionable paper to be published.
The moral here: if you sign it, it's yours.
I have it on good authority that publication in a name journal requires a "name" co author. So Mr. LaCour enlisted Dr. Green who lent his name to the study with little or no oversight on his part.
One of the thing I have learned in life is that when I sign something I take responsibility for it. So when Dr. Green signed the paper, he took co-responsibility for it's contents. But he utterly failed to make any attempt to check the correctness of the application of the data to the results, and therefore permitted a questionable paper to be published.
The moral here: if you sign it, it's yours.
133
He felt it was 'a delicate matter' to ask any questions. What on earth is that about? Is it a retrospective excuse or was he really intimidated by the LaCour?
6
It's not the case that "*the* problem here is with Dr. Green." Surely there is a problem with Dr. Green's lack of rigorous review prior to signing on. But that is only one of several problems that this case illustrates.
A member of my wife's side of the family is a social scientist, and considered one of the foremost experts in his field. He often seems to let his accomplishments go to his head, displaying a shocking immodesty, and zero tolerance for others who aren't as smart as him. He's a good guy, but can be a bit insufferable at times. So thanks to this debunked study, I'm looking forward to reminding him that academics are human beings just like the rest of us. ;)
15
This is not a study about gay unions. This is study about canvassing methods.
And a cautionary tale about the current academic environment and its publish or perish incentive structure. Maybe with a larger tie to our winner-take-all economy.
But it has no bearing on gay marriage, other than to make it harder to know how to run the most persuasive pro gay marriage campaign. Readers will read the headline and think otherwise.
And a cautionary tale about the current academic environment and its publish or perish incentive structure. Maybe with a larger tie to our winner-take-all economy.
But it has no bearing on gay marriage, other than to make it harder to know how to run the most persuasive pro gay marriage campaign. Readers will read the headline and think otherwise.
25
Senior co-authors not looking at the data or the code is unfortunately common. Part of that is because the training that social scientists get in programming is very minimal - we're pretty much self-taught, and so our code is frequently sloppy and difficult to wade through. Making up data is rare, but I would not be surprised if there were a lot more social science findings that were based on coding errors. We need for code review to be an expected part of social science research, but we're nowhere near that yet.
3
It seems to me that science, in the words of Ronald Reagan, must “trust but verify” those who produce articles for publication. Deliberate malfeasance, such as appears to be the case here, is only one way for published results to be misleading. We also see experimenter error and sampling error, not to mention engrained problems (publication bias, the “file drawer” problem, and so on). But lots of people read the articles. Some of them spot inconsistencies right off the bat (hopefully during peer review, but sometimes afterwards). Others, interested in the field, base experiments on the published results and notice the problems when their experiments don't work as they should have.
The chances of someone being misled is greatest with hot new results—after a while, there will be a greater context for the original papers, either supporting or contradicting them.
Perhaps the Times' ethics philosophers will someday address this question: If a researcher publishes in a little-studied domain fictional results based on what he or she thinks should happen, and then many other researchers produce real experiments that successfully replicate and extend those results, thereby solving various longstanding and important problems in science and technology, should the original researcher when a Nobel in science, or in literature?
The chances of someone being misled is greatest with hot new results—after a while, there will be a greater context for the original papers, either supporting or contradicting them.
Perhaps the Times' ethics philosophers will someday address this question: If a researcher publishes in a little-studied domain fictional results based on what he or she thinks should happen, and then many other researchers produce real experiments that successfully replicate and extend those results, thereby solving various longstanding and important problems in science and technology, should the original researcher when a Nobel in science, or in literature?
3
Donald Green's tenure should be reviewed. The questions he claims to be inappropriate or too delicate to ask are exactly the questions that are asked of participants in scientific papers.
19
We all know the problem. Ultimately, it's our out of control worship of competition and 'productivity', and our adoration of the 'business model' even where it's inapt, as in universities. And Science and other major journals are deep into the self-promotion, PR spinning game. The 24/7 'news' environment means that journalists, even those adequately trained, are pressured to hype any little thing that comes along that's curious, cute, or plays on fears (like disease). But self-reinforcing sensationalism by scientists and media (and funders like NIH and NASA) alike is not the deeper source of the problem. Research funding has led institutions to promote uncontrolled growth, because they have a material stake in that, which means too many students are being trained by the unconstrained system, in turn meaning there aren't enough jobs available for them when they graduate.
Nobody can be surprised at what's happening. It's only human. But is it controllable? Who has the will to do that?
Nobody can be surprised at what's happening. It's only human. But is it controllable? Who has the will to do that?
6
The most suspicious and unethical thing LaCour did was delete his data. As a researcher there is no need to delete data to protect confidentiality as names or other unique identifiers need not be collected and if it was that specific field could be deleted and not the entire data set. Also wouldn't he want to keep the original data base to further explore and write more articles, assuming the data originally existed. I have data sets dating back decades that I think I might reanalyze although in truth it will never happen.
23
Social "science" is anything but. There isn't one field that isn't influenced by the political leanings of the "researchers". It doesn't matter if it is economics, sociology or any of these other pseudo-sciences. What makes them so dangerous is that their results don't just end up as a Jeopardy question but end up in actual public policy with usually disastrous results as we saw in 2008.
10
I think there is a huge difference between the words science and political or social science. And that terminology has gotten very confused. In the fields of scientific endeavor I. e. Biology, chemistry, physics and all their related offshoots and combinations ...biophysics, biochemistry... Hard evidence of scientific research must be provided. There is no fluff in the work... There is a hypothesis, the hard evidence (research) and a conclusion. Most scientists publish in refereed journals where their work is read and juried by members of the field who are often extremely critical. There are no pats on the back, so to speak... If anything there is harsh and realistic judgement of the work accomplished. Scientists, or those wanting to call themselves scientists who try to sneak by, write a second paper that is a plagerism of their first paper, or do shoddy research are quickly discovered by one or more of the two or three reviewers. Those papers never see the light of day ... They are either rejected or revisions and another review is required.
the non-science world of social, political, etc. scientists...should have the same peer review, if there is to be validity in the work. What happened here?
This scam sets trust in science back decades.
True research scientists are much less interested in personal glory than they are in good and pure research that moves knowledge forward. Sometimes that small step forward takes years or even a lifetime.
the non-science world of social, political, etc. scientists...should have the same peer review, if there is to be validity in the work. What happened here?
This scam sets trust in science back decades.
True research scientists are much less interested in personal glory than they are in good and pure research that moves knowledge forward. Sometimes that small step forward takes years or even a lifetime.
5
Is anyone really surprised? Efficacy is a rare commodity in research, not because researchers are intentionally fudging data - although I'm sure that happens far more than it's reported - rather because they're so intent upon proving they're right so they get more dollars. The grant business is highly competitive. And where there's competition, objectivity goes out the window. It's a physical law. Cynicism is the antidote.
22
Science, polls, and statistics are much like opinions in that they all possess one of the same tendencies . . . they are malleable and can be skewed to form the desired results.
4
People mistrust science because they read stories like this as evidence that science lies to them.
However, this story is evidence of exactly the opposite. Most social scientists are in it for the science, not the prestige. They follow other people's work and check it for consistency. When scientists try to cheat, they get caught.
However, this story is evidence of exactly the opposite. Most social scientists are in it for the science, not the prestige. They follow other people's work and check it for consistency. When scientists try to cheat, they get caught.
13
First of all, let's not confuse social sciences, such as political science, with the physical sciences. Uncertainties are much higher in the social sciences, and it's much easier to skew and misinterpret data. Second, let's not make sweeping generalizations about "shaken trust" brought on by one example of questionable ethics and shoddy oversight. There are thousands of academic researchers out there doing their best to create new, valid knowledge, despite miserable funding situations, pressure to publish, and the corporatization of higher education.
102
"wages of sleep" I suspect you may have napped through your research methods class where you would have learned that, contrary to you belief, there is very convincing scientific data using solid meta-analysis showing the "uncertainties" or lack of reliability is the same in physics as it is in social science research. It is probably easier to validate hard science results through replication than in the social sciences, but I may, like you, be wrong here.
You are quite right. However, these publication pressures are coupled with widespread incompetence in statistics. This combination leads to many more faulty studies than does outright fraud. This has been a complaint for decades. There is, therefore, ample reason not to trust "scientific" publications in certain fields. For one well-publicized and documented recent example of such incompetence, see http://www.nytimes.com/2011/08/09/health/09network.html?_r=1 and http://pages.iu.edu/~rdlyons/pdf/CF-pub-erratum.pdf ("The spread of evidence-poor medicine via flawed social-network analysis").
I'm not a big fan of the numeric social sciences - I find it difficult to boil down human experience into codes - but I don't think you can claim it is easier to skew and misinterpret data in social sciences than in physical sciences. There are uncertainties and skewed data in all sciences! I think numbers are fudged all the time, just not as blatantly as LaCour did (but of course also those who do their best).
1
The title of this article should be "How Social Science Experiments are Manipulated" I am angry with the NYT for disingenuously implicating the entire gay community in their strides toward marriage equality. This is incidental to the issue at hand but main to sound like the main catalyst. Please...
15
So let me get this straight (no pun intended) - our politicians routinely lie to us, our religious leaders routinely lie to us, our media outlets routinely lie to us, our police routinely lie to us, Wall St banksters routinely lie to us...and now we find out our top research scientists routinely lie us. So what is left of the fabric of this our great society if all is built on lies? The facade Oz will soon crumble and the wizard will be revealed.
3
> our top research scientists routinely lie us.
This statement is False, with a capital 'F'. There are a few bad apples among them, but exceedingly few.
This statement is False, with a capital 'F'. There are a few bad apples among them, but exceedingly few.
One wonders if 'political science' and 'social science' are oxymorons. Is there anyone involved that is unbiased and capable of addressing any issue, especially those with high subjective content, with the truth implied by the word 'science'?
4
This is one more case that shows us that the publish or perish ethos, combined with the tremendous rewards associated with a groundbreaking discovery, are a danger to the scientific project. After all, science is supposed to value all data, whether it supports, undermines, or is inconclusive for a particular hypothesis. However it seems unlikely that Mr. LaCour would've been offered a Princeton professorship without that "groundbreaking" Science article on his CV.
11
This scandal is not just about procedural lapses within the scientific community.
As the AAAS reports, the House of Representatives is preparing to pass an appropriations bill that would slash social science funding at the National Science Foundation by 16%. This episode of apparent scientific misconduct unfortunately plays right into the belief in the Republican-controlled House that NSF should concentrate on the "core sciences" (physics, chemistry, biology, etc.) and reduce or even eliminate funding for the social sciences.
As the AAAS reports, the House of Representatives is preparing to pass an appropriations bill that would slash social science funding at the National Science Foundation by 16%. This episode of apparent scientific misconduct unfortunately plays right into the belief in the Republican-controlled House that NSF should concentrate on the "core sciences" (physics, chemistry, biology, etc.) and reduce or even eliminate funding for the social sciences.
2
Where there is probably much more evidence of questionable research would be in the area of climate change research. So much has been presented that is flawed, especially on the subject of root causes, but accepted as truth it is mind boggling. Yet policy is being built on the house of cards. Not only have we been failed by researchers but also the lack of deep scrutiny by journalism. So many lemmings...
3
Sorry, climate change is exactly the opposite. Occasional startling results can represent fraud, sloppy work or even the 1 or 5% of times that well done work is correct only by chance. Every scientist knows that confidence in a result comes only with direct or systematic replication. Climate change is completely irrefutable because of the many thousands of replications and no reasonable person could confuse the certainty on climate change and this one report on canvasing methods.
Between Mr. LaCour's serious lack of integrity, Dr. Green's lack of responsibility, and Dr. McNutt's lack of discernment, this debacle overshadows the integral work of many other graduate students.
For the sake of these people, Dr. McNutt should retract LaCour's dishonest publication.
For the sake of these people, Dr. McNutt should retract LaCour's dishonest publication.
7
It looks like she'd like to, but is giving the author the time and opportunity to ask for it himself. if the author asks for a retraction, the scientific record is much cleaner. It gets ugly when a journal forces a retraction without the author's request. Remember the Wakefield "vaccines cause autism" disaster in The Lancet? How nice it would have been if he'd retracted his own paper instead of making a career out of being the silenced victim.
Totally agreed. The real heroes here are the two Berkeley grad students who figured this all out and published it online open access so quickly.
As a microbiologist (who has focused on staph infections) with over 110 publications in my career I can attest to the pressure not only to publish but to publish in journals (like Science) that have a high "impact factor". And Science does like "splashy" papers. In the staph field several papers published in Science have suffered from non-reproducibility of the work or were just downright wrong. I even had my name removed as an author from one study before it was published (it, too was wrong). Many of us in the sciences maintain a high degree of skepticism about many publications: "when it sounds too good to be true then it probably is."
25
Bias is programmed into margin of error.
If that is too large the science fails.
If that is too large the science fails.
Bias in a statistical sense has NOTHING to do with bias in a political sense. That word in this context does not mean what you think it means. But that's a common undergraduate mistake.
It's interesting, but this incident (or scandal, if you prefer) could shake one's faith in the scientific community or solidify it. Obviously, even the appearance of impropriety in such a widely circulated journal does serious damage to the professional community. Yet at the same time, we see that drive to reproduce results and further refine new findings is what LED to the truth. In other words, while ideally, society is best served when all initial studies are properly conducted and vetting catches bad ones before they are published, the scientific community itself mad the errors known. That pattern is repeated in the now discredited study of vaccines as they relate to autism. By contrast, consider how embarrassing revelations typically play out in other non-scientific communities like politics or religion. Rather than seeking/revealing the truth, the communities tend to protect bad actors until some outside or unexpected force confronts them with evidence too powerful to explain away; priests turned to pedophiles, NSA overreach, dark money campaign financing, etc. These groups are usually perfectly content to conceal any impropriety until the truth is dragged into the public eye (and then they will often try to prosecute those responsible for the revelation).
12
Of course, this type of data manipulation would never be done with anything as important as global warming, oh wait, there is the climategate scandal from East Anglia CRU, which oddly enough, refused to share its temperature data. Seems to be a pattern here.
Seems that left leaning academics are not scientists nor trustworthy.
Seems that left leaning academics are not scientists nor trustworthy.
3
Difference is a great deal of global warming science has been reproduced and has been scrutinized and accepted by the vast majority of climate scientists, with only a few vocal outliers. The red flag here was that the findings were not reproducible.
This is shoddy logic.
Two incidents are not a pattern.
"The case has shaken not only the community of political scientists but also public trust in the way the scientific establishment vets new findings."
A bit of hyperbole here to say the least. I think it is safe to say the public could care less about political science and be totally confused about the way journals vet what they find, since a vet is where dogs and cats go.
A bit of hyperbole here to say the least. I think it is safe to say the public could care less about political science and be totally confused about the way journals vet what they find, since a vet is where dogs and cats go.
6
Meow.
The public will see the words "gay unions" and assume that this means that we shouldn't legalize gay marriage.
Lousy headline, NY Times.
The public will see the words "gay unions" and assume that this means that we shouldn't legalize gay marriage.
Lousy headline, NY Times.
I entered a PhD program in sociology back in 1991. At that time, sociology as a discipline was in the midst of a transformation from a mostly qualitative endeavor to a quantitative one. All the best sociology departments had shifted their research emphasis from the touchy-feely, vaguely hippyish approaches that had dominated the field in the 60s and 70s to a supposedly more rigorous, data-driven approach that was intended to turn sociology into a "serious" discipline that could "prove" its findings rather than just claim them. I had majored in anthropology in college and found this obsession with surveys and questionnaires and data pretty tedious and a little naive, as if data were impervious to human error or even intentional misrepresentation. We grad students who dared challenge this data-driven mindset were castigated and ostracized within the department. I eventually left without my degree. And yet we've since seen quantitative studies, in many fields, that don't hold up to prolonged scrutiny. We've seen prestigious, peer-reviewed journals seduced (snookered?) by data-dependent articles that make claims that can't be substantiated. So it would seem that quantitative study is not the miracle they all thought it was going to be and a lot of the "hippy-dippy" qualitative work done before this data feeding frenzy has actually held up beautifully. It would behoove academics in social science to stop trying to be physics and accept the real nature of their field.
20
“Michael said he had hundreds of thousands in grant money, and, yes, in retrospect, I could have asked about that,” Dr. Green said. “But it’s a delicate matter to ask another scholar the exact method through which they’re paying for their work.”
This stands out as absurd. The careers of junior researchers are judged by publications (both how many publications and in what journals) and by grant money gained (both how much and from whom). Every academic CV has a section listing all grants including the amounts and the sources. All Prof. Green needed to do was to look at Mr. LaCours's CV and he would have discovered what money Mr. LeCour had obtained and from whom. To say that he didn't do this because it would have been indelicate sounds a loud alarm about Prof. Green's involvement or lack thereof. Surely before taking on a young researcher as a mentor, one would look at the CV of the person asking to be mentored. This statement doesn't pass the most basic test for credibility.
This stands out as absurd. The careers of junior researchers are judged by publications (both how many publications and in what journals) and by grant money gained (both how much and from whom). Every academic CV has a section listing all grants including the amounts and the sources. All Prof. Green needed to do was to look at Mr. LaCours's CV and he would have discovered what money Mr. LeCour had obtained and from whom. To say that he didn't do this because it would have been indelicate sounds a loud alarm about Prof. Green's involvement or lack thereof. Surely before taking on a young researcher as a mentor, one would look at the CV of the person asking to be mentored. This statement doesn't pass the most basic test for credibility.
53
I didn't understand that either. Why wouldn't someone above the researcher -- the "boss," as it were, want to ask his subordinate a question? That's how a layman is going to read this.
It doesn't make sense.
It doesn't make sense.
Right on point. Medical researchers are now REQUIRED to list their funding sources.
Let alone the fact it's very rare for someone so junior to have ANY external funding, so 'hundreds of thousands' strains all credibility. Also, knowing the source is very important - was it the National Science Foundation or an entity with a specific political agenda?
1
To determine whether this study is valid is like voting whether or not 90 degree weather is hot or not.
4
ho-hum... shoddy and incomplete, if misstated data??.. in a way - things worked.. he got busted. unfortunately - it is all too common - and often sadly, in some pharmaceutical 'trials'..where real harm is possible.
8
Dr. Green was his adviser so it all falls on his shoulders, he was not doing his job...apparently at all.
That is a huge problem with graduate schools all over the country, professors who just refuse to do their job and leave graduate students hanging until the last minute and then just take anything the student turns in.
The problem with training solo is that it embeds errors and that is what these graduate students are doing, basically training themselves.
I would need a small book if I recounted all of the horror stories I have heard from gradate students about professors who agree to be advisers then refuse to return calls or e-mails or even show up for appointments. Deadlines are missed because the professor would not sign off on a paper or project then it is the student who is left wondering if they will ever graduate and they have huge student loans that are accruing interest staring them in the face. Is it any wonder that they feel pressured to turn in incomplete work?
In this case though it sounds like the student misused is grant money and instead of paying the participants he kept the money for himself. That is the adviser's fault, he should have kept track of what his student was doing, he was NOT an equal, he was not a fellow scientist, he was a student! That adviser is just trying to cover his tracks now because no one with any sense would fall for the line he is selling.
That is a huge problem with graduate schools all over the country, professors who just refuse to do their job and leave graduate students hanging until the last minute and then just take anything the student turns in.
The problem with training solo is that it embeds errors and that is what these graduate students are doing, basically training themselves.
I would need a small book if I recounted all of the horror stories I have heard from gradate students about professors who agree to be advisers then refuse to return calls or e-mails or even show up for appointments. Deadlines are missed because the professor would not sign off on a paper or project then it is the student who is left wondering if they will ever graduate and they have huge student loans that are accruing interest staring them in the face. Is it any wonder that they feel pressured to turn in incomplete work?
In this case though it sounds like the student misused is grant money and instead of paying the participants he kept the money for himself. That is the adviser's fault, he should have kept track of what his student was doing, he was NOT an equal, he was not a fellow scientist, he was a student! That adviser is just trying to cover his tracks now because no one with any sense would fall for the line he is selling.
8
Professor Green was NOT his adviser! They are at separate universities on opposite sides of the country. Green was a co-author, which of course brings up an entirely different set of factors.
La Cour's actual adviser at UCLA, by the way, authors an occasional column for this very newspaper in the Upshot section.
La Cour's actual adviser at UCLA, by the way, authors an occasional column for this very newspaper in the Upshot section.
“Michael said he had hundreds of thousands in grant money, and, yes, in retrospect, I could have asked about that,” Dr. Green said. “But it’s a delicate matter to ask another scholar the exact method through which they’re paying for their work.”
Dr. Green's reverence for a graduate student colleague simply beggars belief.
Dr. Green's reverence for a graduate student colleague simply beggars belief.
25
Excellent article and one that perhaps brings to light long-standing suspicion of academic publishing and review procedures. In so many professional endeavors a "conflict of interest" principle is a key to objectivity and credibility. This seems to be lacking when publishers, reviewers and academic institutions all consider themselves to benefit from an ever-increasing number of studies being successfully completed, reviewed and published.
4
We have ourselves to blame. Political Correctness has become the law of the Academic land... and justifies any transgression.
8
And such transgressions don't occur in conservative research? If you believe so, the wool is way over your eyes.
1
This article refutes your own statement. "Political correctness", a perjorative term as used by you, would, according to you, have shielded this publication from scrutiny or criticism. In fact, just the opposite has occurred. But don't let that stand in the way of you throwing around your insult toward "political correctness".
2
In our increasingly distrustful age, science continues to be trusted as a guide to accurate observations about the world around us. When the scientific method is rigorously applied and the outcomes are reproduced many times over there is justification for this trust. The method becomes less reliable, though, when wielded by those who have a high stake in a specific outcome. The temptation to adjust the results to achieve personal gain, such as prestige and positions, is too strong for many.
The peer review system is very much needed, but it might function even better if those responsible for vetting the work of other scientists come from a large variety of ideological disposition and not just the decreasing number of acceptable scientific positions that earn tenure and publication agreements in our era. We all live in the tension between what is really happening and what we would like to happen based on our beleifs and values. Letting science reveal to us what is really happening is as hard as the editorial staff of the NYT objectively reporting on scientific claims that undermine their editorial commitments. This article proves it can be done.
The peer review system is very much needed, but it might function even better if those responsible for vetting the work of other scientists come from a large variety of ideological disposition and not just the decreasing number of acceptable scientific positions that earn tenure and publication agreements in our era. We all live in the tension between what is really happening and what we would like to happen based on our beleifs and values. Letting science reveal to us what is really happening is as hard as the editorial staff of the NYT objectively reporting on scientific claims that undermine their editorial commitments. This article proves it can be done.
3
This has nothing to do with "gay unions" and everything to do with a lack of supervision in an academic study.
Some people are going to do nothing but read the headline and infer that oh, there was some study saying good things about gay people and now it's been found that it's a fraud.
This was actually a "maligned study" on politics and voting. It could be any kind of voting. But of course "gay unions" made it a eye-catching internet "click bait" subject to place on the headline of the NY Times and I, too, read it.
I'm not happy that I did, however.
Some people are going to do nothing but read the headline and infer that oh, there was some study saying good things about gay people and now it's been found that it's a fraud.
This was actually a "maligned study" on politics and voting. It could be any kind of voting. But of course "gay unions" made it a eye-catching internet "click bait" subject to place on the headline of the NY Times and I, too, read it.
I'm not happy that I did, however.
86
"...there was some study saying good things about gay people and now it's been found that it's a fraud." Well, isn't that the intent of this article?
To me, this tempest in a teapot is being blown way out of proportion. I mean, who exactly was harmed by this other than the researchers? Meanwhile, the anti-gay side is using false data to support arguments that have far more real implications for far more real people -- specifically, they want to take children away from their gay parents. Where is the outrage about that? Where are the NYT headlines? Good luck finding any -- there are none.
To me, this tempest in a teapot is being blown way out of proportion. I mean, who exactly was harmed by this other than the researchers? Meanwhile, the anti-gay side is using false data to support arguments that have far more real implications for far more real people -- specifically, they want to take children away from their gay parents. Where is the outrage about that? Where are the NYT headlines? Good luck finding any -- there are none.
1
I'm surprised you don't see the harm in this. It tarnishes the journal Science, the co-author Dr. Green, and the very concept of social science research. It also means a pretty important job, that of Princeton professor, was given to a fraud over other candidates. The scandal itself doesn't really have anything to do with the LGBT community.
Also, you are suggesting that lying about research is OK as long as it supports your own opinions. For research to have any validity or impact, it must be fueled by the pursuit of truth over ideology.
Also, you are suggesting that lying about research is OK as long as it supports your own opinions. For research to have any validity or impact, it must be fueled by the pursuit of truth over ideology.
Regardless of the issue, given the professional stakes and ego involved, a certain proportion of researchers will fake their results. And the paradigm is that the scientific community will impartially vet such things and prevent the dissemination of such falsehoods.
And while I believe there is a predisposition within the scientific community (as in any other) to presuppose the validity of studies basically confirming that which that community prefers to believe, we should also note in fairness that, in this case, that system worked because the fraud was detected and exposed. So while sloppy and/or deliberately fabricated research can and will be produced, this makes the point again how crucial it is for scientific peer review to be thorough and impartial. For as long as that is, we may find the occasional blip, but over time, science will get it right. And if the scientific community ceases to be impartial, refusing to challenge those positions more comfortable to its wishes, science will descend to nothing more than politically correct propaganda.
And while I believe there is a predisposition within the scientific community (as in any other) to presuppose the validity of studies basically confirming that which that community prefers to believe, we should also note in fairness that, in this case, that system worked because the fraud was detected and exposed. So while sloppy and/or deliberately fabricated research can and will be produced, this makes the point again how crucial it is for scientific peer review to be thorough and impartial. For as long as that is, we may find the occasional blip, but over time, science will get it right. And if the scientific community ceases to be impartial, refusing to challenge those positions more comfortable to its wishes, science will descend to nothing more than politically correct propaganda.
15
“I thought it was a very ambitious idea, so ambitious that it might not be suitable for a graduate student,” said Dr. Green...
And yet, Dr. Green signed on and generously handed over his stellar reputation to Madoff style sham-researcher who has mysteriously erased the raw data? Mr. La Cour (what's in a name?) should go the way of Jason Blair (a life coach!) and Steven Glass, unless, of course, Princeton needs a mascot.
And yet, Dr. Green signed on and generously handed over his stellar reputation to Madoff style sham-researcher who has mysteriously erased the raw data? Mr. La Cour (what's in a name?) should go the way of Jason Blair (a life coach!) and Steven Glass, unless, of course, Princeton needs a mascot.
50
I believe this is the third time I have read about this study in the Times. The first soft peddled the fraud by juxtaposing it with Mark Regnerus's study on gay parents which unlike this one remains valid because he did not defraud his very partisan funder. This appears to me a back handed attempt to salvage some integrity for the disciplines of social and political science and for that matter gay activism.
I remember when I first read about and then dismissed this study on it's face. It really was just to fancifull -as opposed ambitious which is how Dr.Green apparently desperately wants it to be thought of. The very best social surveys boast a rate or "replicablility" of less than fifty percent and it is with this in mind that I and anyone I spoke to about it concluded it was just bonkers, just way too many moving parts, This may come as a great shock but the implicit claim of this piece that Science or any publication funcitions as a benchmark is just not true- not any more.
I am not a big fan of the gay lobby in fact I find a good deal of their methods immoral and most of the rest unethical but with regard to this study they appear to have been an unwitting dupe who upon realization of this unhappy state did the right thing. This in my book is no small thing so perhaps something good has come of the whole affair.
I remember when I first read about and then dismissed this study on it's face. It really was just to fancifull -as opposed ambitious which is how Dr.Green apparently desperately wants it to be thought of. The very best social surveys boast a rate or "replicablility" of less than fifty percent and it is with this in mind that I and anyone I spoke to about it concluded it was just bonkers, just way too many moving parts, This may come as a great shock but the implicit claim of this piece that Science or any publication funcitions as a benchmark is just not true- not any more.
I am not a big fan of the gay lobby in fact I find a good deal of their methods immoral and most of the rest unethical but with regard to this study they appear to have been an unwitting dupe who upon realization of this unhappy state did the right thing. This in my book is no small thing so perhaps something good has come of the whole affair.
11
You're not a "big fan" of the "gay lobby" and find our methods "immoral?" What part of equality is immoral?
5
I felt the same way. I have never known any canvassers to be great in changing people's opinions. It is rare even to change a friend's opinion, even if you ply them with wine and a good meal. You might be able to sell a stranger life insurance, but you're not going to change the way they think about sexuality.
3
What methods are 'immoral?'
1
I work with scientific researchers, and I can attest that "publish or perish" causes numerous lapses in ethics. That happens even more in the "social sciences," since that data is hard to reproduce anyway. It is rare, however, for people to be blatantly dishonest, as it appears happened here. This type of fraud is caused by two forces:
1) desire to grab headlines,
2) a political agenda supported by a powerful lobby.
This fraud is a warning shot to our society on the limitations of scientific research, especially if it is supported by a group with a clear political agenda. People are immediately skeptical of cancer research funded by a pharmaceutical company, and they should be even more skeptical for research connected to the latest cause célèbre (e.g. gay rights, sexual assault on campus, climate change, racist cops, inequity, etc).
1) desire to grab headlines,
2) a political agenda supported by a powerful lobby.
This fraud is a warning shot to our society on the limitations of scientific research, especially if it is supported by a group with a clear political agenda. People are immediately skeptical of cancer research funded by a pharmaceutical company, and they should be even more skeptical for research connected to the latest cause célèbre (e.g. gay rights, sexual assault on campus, climate change, racist cops, inequity, etc).
33
In your criticism of scientific research you fail to note that the scientific method worked in this case. The last step in the scientific method that every high schooler learns is to repeat the experiment to insure the results are valid. In this case, and any other studies that are published, this was done and the results fell apart. Whether by poor design, faulty methods, inaccurate interpretation of the data, or, in this, possible fraud, if the results can't be repeated they should be rejected. Just because science produces results that conflict with your personal beliefs it doesn't justify calling into question the validity of those results.
I would think that from the grad student's perspective, there was also another likely agenda: getting a Science article could make your career before you even start it.
1
Claiming that the scientific method succeeded in this case because fraud was detected is like saying the criminal justice system "succeeded" after a convicted person is later found to be wrongly convicted. No, it's a failure in both cases. The right questions and considerations were not given in the first place and that is failure regardless of how many roses you place around it to help the smell.
2
As a social science researcher, I think Green's behavior is as problematic as LaCour's. Not sharing raw data would be a big red flag for me and make me suspicious of LaCour's motives. Skepticism should have kicked in much earlier. What research would get a "huge" grant to do research and not reveal the source? Getting a big grant is big brag in academia and revealing the source of funding is the first step in understanding a possible bias to the study.
Its too bad this kind of dishonesty casts a pall on all of us who study the social world. Peer review has flaws that can be exploited and it appears this is one time it got caught.
Its too bad this kind of dishonesty casts a pall on all of us who study the social world. Peer review has flaws that can be exploited and it appears this is one time it got caught.
83
I agree. In 1997 as a doctoral candidate I was awarded a Fulbright for research overseas. The entire amount was $18,000 for the year. Certainly many studies receive many thousands or even hundreds of thousands but these tend to be granted to groups of well-known researchers and rarely for social science work. This is not for a super-collider that will find a new particle to change the world of science. "Hundreds of thousands" according to a grad student describing the funding for his research not only should have run alarm bells, it is simply absurd.
But the article, like the editorial the other day here, is right to put some of the blame not only on the perennial, but now stronger-than-ever "publish or perish" but also on the new mantra of academic hiring committees "impact factor" or publishing in "high impact" journals. What the article misses is that this comes in part from the greater commercialization of universities which are expected to "be run like businesses" and so must demonstrate "productivity" on the part of faculty members. Since it is impossible to demonstrate productivty in teaching, which is qualitative by nature, no one cares anymore if you can teach (though being "evaluated" is important, but again only through quantifiable results), but the number of publications can be quantified, but the concept of "impact" measures how many other academics cite your work. This whole intellectual circle jerk - now passes for serious intellectual endeavor.
But the article, like the editorial the other day here, is right to put some of the blame not only on the perennial, but now stronger-than-ever "publish or perish" but also on the new mantra of academic hiring committees "impact factor" or publishing in "high impact" journals. What the article misses is that this comes in part from the greater commercialization of universities which are expected to "be run like businesses" and so must demonstrate "productivity" on the part of faculty members. Since it is impossible to demonstrate productivty in teaching, which is qualitative by nature, no one cares anymore if you can teach (though being "evaluated" is important, but again only through quantifiable results), but the number of publications can be quantified, but the concept of "impact" measures how many other academics cite your work. This whole intellectual circle jerk - now passes for serious intellectual endeavor.
5
@James Wittebols in Detroit, MI: I must disagree with you, Mr. Wittebols. It appears Michael LaCour fabricated data, in other words, just made it up out of nothing. Professor Green is guilty of lax oversight of Mr. LaCour's work. Fabricating data out of thin air is the worse research crime here. We could prove anything we wanted to if we were allowed to invent the data.
He was a co-author. Yes LaCour is more dishonest by far but Green should have seen his name being used here and been more involved as a co-author.
2
Science, like any other institution tends to fall in line with the prevailing currents and the currents these days seem highly political verging on authoritarian/totalitarian.
8
Politics always hijacked science when it was convenient. 80 years ago the scientific consensus was that the white race was superior to all other races. That view suited the politics of the day, but was not scientific. We have the same nonsense with global warming today as that issue is a darling of the political class.
Sorry but it is quite the opposite: it is the pressure to conform to market forces and to show "productivity" as if universities could be run like businesses that has changed academia from a place where colleagues evaluated your work as a teacher and member of a community and where in exchange for not making the kind of incomes other professionals make you got tenure so you could keep doing that work, to having to demonstrate that you were productive. Doesn't a professor having 100 publications seem excessive to you? Doesn't it suggest that some of those articles and books likely re-pass the same research material merely to demonstrate continued publishing? But universities are under pressure to show that they can be run like businesses and hiring committees now insist on publications en masse in impact factor journals plus the demonstrated ability to bring in lots of (corporate mostly) funding from outside for that research. None of this has anything to do with learning.
1
In this day and age, no one should publish research without the raw data and the code used to produce the results. Data can be anonymized with affecting the outcomes, if need be.
The good news is that in the long run, good science wins out. There will always be people who try to cheat, but they do get caught.
The good news is that in the long run, good science wins out. There will always be people who try to cheat, but they do get caught.
53
I agree, eventually they get caught. Look at the global warming hoaxers like East Anglia CRU and Michael Mann. Tried to hide all the raw data and the manipulations they did, but fortunately a good citizen exposed their fraud. But what happened? The scientific community and the political community tried to rally round and protect them.
"Data can be anonymized with affecting the outcomes, if need be."
I don't think so. For data to be truly validated, *fully* validated, if need be; then each individual piece needs to be traceable to it's source. How else can you prevent fraud if you cannot gain access to the source to confirm?
I don't think so. For data to be truly validated, *fully* validated, if need be; then each individual piece needs to be traceable to it's source. How else can you prevent fraud if you cannot gain access to the source to confirm?
1
The problem as I see it as that many of these studies are ultimately topical and so are never repeated so "good science" does not necessarily win out. Instead, too often, "bad scientists" are incentivized to cut corners to come up with interesting topical research findings.
1
It is important to note that it was the scientific community that corrected itself in this instance. The system worked.
I think the general rule is that you need to wait a few years (and hopefully a few extra studies) before you believe a result. This applies to all research, be it faster than light neutrinos, gravity wave detection, nutritional studies, miracle drugs, etc. It's only when a theory has been vetted time and time again and there's a general scientific consensus that you can say a theory works. Quantum mechanics is one of the most tested theories ever devised. Evolution has proven itself with just about every fossil record and lab result. Climate change has so much proof from so many different directions that it has achieved a scientific consensus on the subject.
Yes, there is the occasional fraudulent paper, but in the grand scheme of things this result will be discarded and researchers will continue advancing knowledge.
I think the general rule is that you need to wait a few years (and hopefully a few extra studies) before you believe a result. This applies to all research, be it faster than light neutrinos, gravity wave detection, nutritional studies, miracle drugs, etc. It's only when a theory has been vetted time and time again and there's a general scientific consensus that you can say a theory works. Quantum mechanics is one of the most tested theories ever devised. Evolution has proven itself with just about every fossil record and lab result. Climate change has so much proof from so many different directions that it has achieved a scientific consensus on the subject.
Yes, there is the occasional fraudulent paper, but in the grand scheme of things this result will be discarded and researchers will continue advancing knowledge.
213
Well, there is a difference between not being able to reproduce a result from a study that might have been a one off, thus, not as compelling as first thought -- and just making stuff up. Of course, the study might have been interesting but ultimately wrong, and that would be okay -- what would not be okay is if data was not collected or if it was manipulated so that the results were actually false. The first does not require retraction but clarification or refinement. The second requires retraction.
1
How is political science a science?
41
It is a social science that uses data to draw probable conclusions. Thus, it utilizes the scientific method. Of course, it's not considered 'hard science' and nobody has ever made that claim.
2
In the loose context of "science" versus "art", a science will investigate what already exists in nature, and an art will create something that does not already exist in nature.
1
Speaking as a physical chemist, my answer would be: "it's not."
3
Two questions that are shamefully unasked: where did all that money come from, and is the Times reviewing its own reporting?
43
The question not asked is, "Where did the money go? If the "paid" surveyors did not get paid and the survey company did not get paid, who ended up with the grant money (assuming there really was a grant)?
1
What money? It appears from this report that the study was not actually done.
2
It seems clear that "all that money" was actually a mirage.
2
Clearly the "scientific" community needs to require raw data to be submitted with all papers as a requirement for published.
Until they do something it brings into question their entire body of work.
Until they do something it brings into question their entire body of work.
11
If all of this is true, it casts a negative on everyone who has done a dissertation.
"If all of this is true, it casts a negative on everyone who has done a dissertation." This makes as much sense as saying if I write a bad check it casts a negative on everyone with a checking account.
14
Really? Every mistake made in a profession does this? How about the police? Can we call them all thugs now, with the recent cases of killing unarmed people?
3
This is a very sad situation, especially inasmuch as, if we are to judge by the methods used to push forward the referendum in Ireland, the basic premise may well have had merit. Door to door canvassers for marriage equality reportedly played a key role in changing minds in Ireland. Of course the situation in Ireland is not exactly comparable to the U.S. Ireland is essentially a uni-cultural society with, at present, a serious grudge against the leadership of the Catholic Church. Still, the Irish methods do raise the strong possibility that door-to-door, face- to -face canvassing is a worth while technique. Unfortunately we won't know from the LaCour study. One hopes a serious and rigorous will replace it. We in the LGBT community, as well as others, need to know.
5
The question wasn't whether canvassing works, but whether canvassing by those directly affected by an election's outcome is more effective than by those who are interested in the issue but not being directly affected.
1
Political science is not science
39
Explain to me using hard data that you cite why a leftist party won the elections in Greece but the right wing won in Poland and Britain. Or explain what factors led Latin America to vote pro-market forces in to power throughout the 1980s but leftist forces in most countries ever since. Or how about why South Korea, which was poorer than Ghana in 1950 is now rich but Ghana is not. Or why India has maintained democracy but Pakistan and China have not. Or whether a country that has a market economy will eventually become democratic and if so why that hasn't happened in China. People who spout that political science is not a science mean that their own opinions on politics are not scientific. And they are right. This has nothing to do with serious work.
4
Steven,
That's a tall order for 1500 characters! "Opinions" are not "science" until there is data to support them. Selecting data that supports what you already think you know is not science either.
"Science" is other people being able to duplicate your results and data. This means that you must make the data you collected as well as information on how it was collected available to those trying to duplicate your work.
When the subject is contentious or political the researchers often won't do this. Take "Climate Change" as an example. Only a fool would state that the Earth's climate doesn't change. The seas were 300' lower and the northern hemisphere was covered with ice just 20,000 years ago! The contentious and political issue is how much the billions of humans alive today affect that existing rate of change and whether short of reducing the human population dramatically anything else will moderate the current rate of change. By the way there is yet no scientific proof that reducing the population will affect the rate of change either.
We don't have the data necessary to even understand why the climate changes in the first place! None of this stops people from having opinions and "remedies" based on those opinions backed by selected data. That is "political" and something the NYT is excellent at doing. The "proof" is in, we are right! So stop looking at the data and do as we tell you! (not!) or you're a "Troglodyte"! Social Science is very much like that too!
That's a tall order for 1500 characters! "Opinions" are not "science" until there is data to support them. Selecting data that supports what you already think you know is not science either.
"Science" is other people being able to duplicate your results and data. This means that you must make the data you collected as well as information on how it was collected available to those trying to duplicate your work.
When the subject is contentious or political the researchers often won't do this. Take "Climate Change" as an example. Only a fool would state that the Earth's climate doesn't change. The seas were 300' lower and the northern hemisphere was covered with ice just 20,000 years ago! The contentious and political issue is how much the billions of humans alive today affect that existing rate of change and whether short of reducing the human population dramatically anything else will moderate the current rate of change. By the way there is yet no scientific proof that reducing the population will affect the rate of change either.
We don't have the data necessary to even understand why the climate changes in the first place! None of this stops people from having opinions and "remedies" based on those opinions backed by selected data. That is "political" and something the NYT is excellent at doing. The "proof" is in, we are right! So stop looking at the data and do as we tell you! (not!) or you're a "Troglodyte"! Social Science is very much like that too!
2
The "feelings thermometer" ? Really?
Everything that is wrong with this study by Mr LaCour is evident in the article. It is extremely difficult to change minds on complex or difficult subjects and this is certainly one of them. What drew the suspicions of the other researchers working on the issue of transgender equality in Florida when they tried to apply the same approach as Mr LaCour and received only a response rate of 3 percent instead of the 12 percent rate claimed by Mr LaCour should have and did raise red flags as to the validity of the study. I once asked a friend of mine if he would officiate at a gay wedding and he said no. He is a very devout Christian and I am reasonably certain that he is tolerant of all but this was simply one threshold over which he would not cross. And no attempt on my part would change his view.
Everything that is wrong with this study by Mr LaCour is evident in the article. It is extremely difficult to change minds on complex or difficult subjects and this is certainly one of them. What drew the suspicions of the other researchers working on the issue of transgender equality in Florida when they tried to apply the same approach as Mr LaCour and received only a response rate of 3 percent instead of the 12 percent rate claimed by Mr LaCour should have and did raise red flags as to the validity of the study. I once asked a friend of mine if he would officiate at a gay wedding and he said no. He is a very devout Christian and I am reasonably certain that he is tolerant of all but this was simply one threshold over which he would not cross. And no attempt on my part would change his view.
21
So if this is a peer-reviewed paper in a major journal what are we to conclude about the unending papers on The WarmList which are generating much needed credentials for our university faculties?
3
This headline was very misleading. The study wasn't on gay unions - it was on effectiveness of a certain canvassing technique to alter opinions about laws on gay unions.
253
Not only that, but it seems to show the ability of the scientific to police itself and try to proceed in good faith with its work. Can you imagine an auto manufacturer or wall street banker doing the same?
11
Agreed. And the accompanying photo of the two men might have suggested, at a glance, a same sex couple. That the study was about gay marriage attitudes was really just a side issue. The Times, as great a newspaper as it is, may need to review its headline policies. I remember hearing about this study a few weeks ago on NPR and thought its ultimate "finding" was remarkable, almost unbelievable. Maybe it was.
10
I understand your point. But I wonder what headline you'd suggest.
As a former social science researcher myself, this smells so extremely of scam.
The damage extends beyond the journal Science and calls into question "ground-breaking" studies of all types.
The damage extends beyond the journal Science and calls into question "ground-breaking" studies of all types.
45
In deed it does.
2
My faith in the editors of Science has taken a dip, but not in the overall scientific community. Imperfect as it is, being made up of humans, it appears to be functioning well enough.
It isn't just Science. It seems that a lot of the problems with people's trust in scientific methods and the scientific enterprise arise from over-reporting, early reporting, poor reporting, and misunderstanding of their topics by writers and editors working against deadline. We would be better off with less but better science reporting -- scratch that: with less but better reporting overall. We don't need more information, we need better information, about everything.
It isn't just Science. It seems that a lot of the problems with people's trust in scientific methods and the scientific enterprise arise from over-reporting, early reporting, poor reporting, and misunderstanding of their topics by writers and editors working against deadline. We would be better off with less but better science reporting -- scratch that: with less but better reporting overall. We don't need more information, we need better information, about everything.
180
The headline of this article is an example of the problem: Whose trust is being shaken? The participants'? The public's? The people directly involved in the story? All of the above? Where are *your* data about how this mistrust is manifested?
How much influence does this particular incident have on the practice of scientists, and how does it differ between social and "hard" scientists? Are submissions to Science falling off? Are academic departments rethinking their reliance on publication history in their hiring considerations? Are major science publications changing their oversight policies?
The article itself seems solid and well done, but the misleading headline shakes my trust in the NYT, not for the first time, which is sad and frustrating.
How much influence does this particular incident have on the practice of scientists, and how does it differ between social and "hard" scientists? Are submissions to Science falling off? Are academic departments rethinking their reliance on publication history in their hiring considerations? Are major science publications changing their oversight policies?
The article itself seems solid and well done, but the misleading headline shakes my trust in the NYT, not for the first time, which is sad and frustrating.
125
I just noticed more: "Maligned" means "to speak harmful untruths about; speak evil of; slandered; defamed."
Good grief, NYT, use the dictionary! And change the headline, please. You still have time.
Good grief, NYT, use the dictionary! And change the headline, please. You still have time.
36
Social "scientists" is oxymoronic, really, a clumsy effort to reduce quality to quantity. That huge flaw even applies to some of the so-called "hard" sciences. Scientists rely upon a huge amount of presuppositions before they can even structure an experiment. Naturalism and materialism simply cannot be the foundation of science. Theism is. Ask Antony Flew, the ferocious former atheist turned theist. [Oops. Too late.]
1
My favorite was a study in a leading mental health journal on 4 men with AIDS. One subject was sexually involved w/another. I was apoplectic! FOUR subjects a valid study do not make. If one subject was sexually involved w/another the "study" has absolutely NO validity even as a curiosity!
I was careful, doing a paper on popular views & attitudes on AIDS, to draw no conclusions espec. since there was disagreement by a professional colleague about what conclusions could be drawn.
5 years or so previously, as a newly diagnosed diabetic, I spent months researching diabetes, reading articles from medical/scientific/social sciences journals. I was appalled then at the garbage: jargon, plethora of acronyms, glaring errors in major medical journals. The more things change...
If it sounds too good, it probably is. If it smells like equine excrement, it probably is. It's still publish or perish.
We have friends who deliberately respond to surveys CONTRARY to their beliefs to confound the data! GIGO!