In 2023, we wrote a series of posts describing evidence of fraud in four articles co-authored by Harvard professor Francesca Gino. The four papers were retracted and, after Harvard conducted a lengthy investigation that found direct evidence of data tampering (see, for example, Colada 118: “Harvard’s Gino Report Reveals How A Dataset Was Altered”), Gino was fired [1].
In our Introduction to those posts (.htm) we emphasized that our concerns were not limited to those four papers, expressing our belief that many more of her papers contain fabricated data. And so, here we are again, with a fifth post, describing evidence of fraud in another article co-authored by Gino.
Specifically, this post concerns a 2011 article by Adam Grant, Francesca Gino, and David Hofmann entitled “Reversing the extraverted leadership advantage: The role of employee proactivity” (.htm). Adam Grant, the paper’s first author, sent us two data files: a Gino-authored file containing the published data, and an earlier file obtained from a research assistant who helped run the study [2]. In this post, we simply compare those files to see how the data were changed [3].
Fold Me Once…
In this article, which has more than 900 Google Scholar citations, the authors hypothesized that extraverted leadership increases productivity when employees are passive, but decreases productivity when employees are active. It contains two studies. Study 1 was correlational and not analyzed by Francesca Gino, while Study 2 was experimental and analyzed by Francesca Gino [4]. We will focus on Study 2.
Study 2 was conducted at the University of North Carolina in 2010, where Gino was employed as a faculty member [5]. Four- or five-person teams had 10 minutes to fold as many t-shirts as they could. Each team had one person assigned to act as an extraverted or introverted leader, two assigned to be followers, and one or two confederates assigned to model either active or passive behavior [6]. As predicted, teams folded more shirts when the team had an extraverted leader and passive followers, or an introverted leader and active followers. See their results figure below [7]:
Comparing The Gino Data File to The Earlier Data File
The “Earlier” data file – obtained from a research assistant – was created earlier (March 29, 2010, by a user called “chip”) than the “Gino” data file (April 20, 2010, by “Francesca Gino”) [8]. The Earlier file contains the research assistant’s counts of the number of t-shirts folded by each participant. The Gino file also contains these counts, as well as some additional data, including participants’ subsequent responses to a follow-up Qualtrics survey [9]. Our analyses focus on the counts, since the counts (1) appear in both files and (2) represent the study’s primary dependent measure [10].
Before comparing the files, we verified that the Gino file does in fact produce the published results (see Footnote: [11]).
The Gino file contains data from four participants who were not in the Earlier file, leaving us with 159 participants (out of 163) who have data in both files [12]. Of those 159 participants, most (130) have exactly the same t-shirt counts in both datasets. However, for 29 participants (18.2%) the t-shirt counts are different between the two datasets.
If these discrepancies represent random errors, then they should be…random. But if they instead represent data tampering, then they should be overwhelmingly in the direction of the study’s hypothesis.
As shown in the figure below, they are overwhelmingly in the direction of the study’s hypothesis:
This figure plots t-shirt counts in the Earlier data against t-shirt counts in the Gino data. If all the counts were the same, then all the dots would be on the diagonal line. But, as noted above, there are 29 that aren’t. For example, the leftmost dot shows a participant with 21 folded shirts in the Earlier data, but 61 t-shirts in the Gino data. Their fold count was increased by 40.
The figure shows that all but one of those changes benefit the authors’ hypothesis. Participants hypothesized to perform well (the blue dots) have much higher t-shirt counts in the Gino file than they do in the Earlier file. Participants hypothesized to perform poorly (the red dots) have much lower t-shirt counts in the Gino file than in the Earlier file. The lone exception represents a change that is as small as it can be: a red 39 in the Earlier file became a 40 in the Gino one.
It’s also worth noting that the changes weren’t small and haphazard, but often large and round. Indeed, 25 of the 29 changed t-shirt counts differed by at least 10 (and 10 of the counts changed by at least 30), and 21 of the changes were by exactly 10, 20, 30, or 40. This is not what you’d expect if the discrepancies arose from innocent measurement errors, reporting errors, or minor corrections.
We can’t imagine a plausibly benign explanation for this pattern. These data were altered to achieve the hypothesized result.
Indeed, if we re-run the authors’ key analysis using the earlier data, the critical interaction goes from highly significant – F(1,51) = 7.16, p = .01 – to nothing at all – F(1,51) = 0.21, p = .65 [13].
This is about as cut-and-dried as it gets. The published data are different from the earlier data in a way that gets the desired result. Our opinion is that this represents the presence of data tampering in another paper co-authored by Gino. It is safe to say that this is yet another Gino-authored study that has been fabricated.
We Need A Broader Audit of Gino’s Research
Detecting data fraud is usually slow and difficult. But not this time. When you have the before and after versions of the data, you don’t need sophisticated data forensics. You just need to compare some numbers.
In 2011, Tilburg University professor Diederik Stapel was caught fabricating data in many different research articles. What happened next? The right and necessary thing: a full-scale audit of Stapel’s research. That audit resulted in a public comprehensive report (.htm) and led to the eventual retraction of 58 Stapel-authored articles (.htm).
We now have overwhelming evidence of fraud in five Gino-authored publications spanning at least nine years. Harvard’s investigation into four of those instances led them to this conclusion:
“On the basis of the evidence gathered and evaluated by the Investigation Committee, the Committee concludes that Professor Gino has engaged in multiple instances of research misconduct, across all four studies at issue in these allegations. Because the papers reporting these studies span eight years in their publication dates, with different co-authors, in different journals, assisted by different lab personnel, and out of different home institutions for Professor Gino, the Committee is concerned about other possible instances of research misconduct in Professor Gino’s studies” (Exhibit 5, p. 41, emphasis added).
Note their concern about “other” studies. If four of Gino’s studies are fraudulent, then probably more than four are fraudulent. This post bumps it up from four to five, but the logic is the same. It almost certainly isn’t just these five papers.
There needs to be a broader audit of Gino’s work, so that all of the fraudulent studies are identified. Fabricated studies impose an insidious cost on other researchers. Researchers waste time reading those papers, even though their results are not real. Researchers try to build on those unreal results because they have no way of knowing not to. And even the researcher who manages to avoid either of those obstacles may find themselves given the impossible task of having to reconcile their own real findings with fabricated findings.
When we wrote this post, it looked like Gino’s ongoing case against Harvard was going to trial next summer. But last week, the judge ruled against her on all eight of the remaining counts. (Here is the judge’s decision: .pdf.) Her case against Harvard is over, while Harvard’s case against her – for allegedly altering a file in an attempt to demonstrate her innocence – remains (see Colada 136). This lawsuit has been hanging over every attempt to conduct the broad audit that we have always known to be necessary. Now that it is over, it is time to get it done.
If there are five fraudulent papers, then there are more than five. Perhaps dozens.
Author Feedback
We shared our analyses and a draft of this post with Adam Grant and David Hofmann. Grant and Hofmann were very responsive and forthcoming. They contacted the Academy of Management Journal – which published the paper in 2011 – and submitted an Expression of Concern. That Expression of Concern is now published but paywalled (.htm), and so they asked us to share it in a form that all readers can access: .pdf.
Footnotes.
- The exhibits of her (now-dismissed) lawsuit against us also show direct evidence of data tampering (see Colada 114), as does much of the lengthy Harvard Report, which was filed as Exhibit 5 in support of Harvard’s motion to partially dismiss Gino’s lawsuit (.htm). As just one reminder, included among this evidence was a private Excel file on her computer (“2012Excel file”) containing highlighted cells of data that were altered (by someone using her computer) in a subsequent published “DAC file” (Exhibit 5, pp. 638-640), which plainly shows evidence of data manipulation. In this table you can see how gray-highlighted values in the earlier file were subsequently changed in the published file (from 0s to 1s):
So, yeah, it seems that the author of these files – the report says it was “Dr. Gino” (p. 634) – identified values that needed to be changed, highlighted those values, and then changed them. Not surprisingly, those changes benefited the authors’ hypotheses (Exhibit 5, pp. 641-642). The whole report is worth reading.[↩] - Grant initially contacted us to seek help assessing the integrity of the data from his collaborations with Gino. He shared Gino’s file and, at our request, subsequently obtained the research assistant’s file. He shared that file with us a few weeks ago.[↩]
- The data and code for this post are available on ResearchBox (.htm).[↩]
- Gino’s co-authors have indicated that Gino was not involved in data collection for Study 1 and that she was involved in data collection for Study 2 (.htm). The Study 2 data file’s metadata indicates that the dataset was Authored by “Francesca Gino”.[↩]
- Hofmann was also employed there.[↩]
- Most teams (51) had two confederates and therefore five members, but five had only one confederate and therefore four members.[↩]
- The experiment’s key analysis – a test of the interaction between leadership condition and follower condition – controls for team size. This control is necessary because teams with more people fold more t-shirts, and all five smaller (four-person) teams were in conditions hypothesized to have lower t-shirt counts: four in the introverted/passive condition and one in the extraverted/active condition. However, the paper’s reported simple effects – the effects of leadership condition within each follower condition – did not control for team size. Adding this control increases the p-value from .011 to .015 within the active condition, and from .021 to .176 within the passive condition. Thus, one of the paper’s two reported simple effects is not significant when controlling for team size.[↩]
- The research assistant’s file contains a participation schedule indicating that the last day of data collection was April 5, 2010. And so it seems that the “Francesca Gino” file was created after data collection was complete.[↩]
- The survey collected responses designed to serve as mediators and manipulation checks, as well as self-reported t-shirt counts (to corroborate the research assistant’s counts). We and Gino’s co-authors don’t have access to this original Qualtrics file.[↩]
- The research assistant told Grant and Hofmann that the Earlier data file may be imperfect, as a number of values may have been corrected between when they were entered on this spreadsheet and when they were more officially entered into Qualtrics. But, as explained below, those presumably small and random imperfections won’t come close to explaining the results we find here.[↩]
-
Reproducing the published findings with the Gino file
Demographics
Folded T-shirt Counts
[↩] - Our analyses require us to distinguish confederates from real participants. We do so by relying on the confederate designations in the Gino file. Confederate designations are missing from six teams in the Earlier data file, and there are 4 participants whose confederate status differed between the Earlier file and the Gino file. It is impossible to know how to interpret these discrepancies. They may represent either innocent corrections or another kind of fabrication. By relying on the confederate designations in the Gino data file, we are treating these discrepancies as innocent. None of this materially influences the results we present in this post.[↩]
- The paper’s reported interaction is F(1,51) = 7.17 instead of the F(1,51) = 7.1648 that we obtain. This tiny difference is very likely due to a rounding difference.[↩]