There is a 2019 paper, in the journal Political Analysis (htm), with over 1000 Google cites, titled "How Much Should We Trust Estimates from Multiplicative Interaction Models? Simple Tools to Improve Empirical Practice". The paper is not just widely cited, but is also actually influential. Most political science papers estimating interactions now-a-days, seem to…
[120] Off-Label Smirnov: How Many Subjects Show an Effect in Between-Subjects Experiments?
There is a classic statistical test known as the Kolmogorov-Smirnov (KS) test (Wikipedia). This post is about an off-label use of the KS-test that I don’t think people know about (not even Kolmogorov or Smirnov), and which seems useful for experimentalists in behavioral science and beyond (most useful, I think, for clinical trials and field…
[119] A Hidden Confound in a Psych Methods Pre‑registrations Critique
A forthcoming paper in Psych Methods (.pdf) had a set of coders evaluate 300 pre-registrations in terms of how informative they were about several study attributes (e.g., hypotheses, analysis, DVs). The authors analyzed the subjective codings and concluded that many pre-registrations in psychology, especially those relying on the AsPredicted template, provide insufficient information., Central to…
[118] Harvard’s Gino Report Reveals How A Dataset Was Altered
As you may know, Harvard professor Francesca Gino is suing us for defamation after (1) we alerted Harvard to evidence of fraud in four studies that she co-authored, (2) Harvard investigated and placed her on administrative leave, and (3) we summarized the evidence in four blog posts. As part of their investigation, Harvard wrote a…
[117] The Impersonator: The Fake Data Were Coming From Inside the Lab
A previous version of this post was supposed to go live in January 2019. But the day before it was scheduled, the Data Colada team (Uri, Leif, and Joe) received an email that we took to be a potential death threat. After discussions with the local police, the FBI, and our families, we decided to…
[116] Our (First?) Day In Court
“Any update on the lawsuit?” That is the most common question any of us is asked. It is usually preceded by an apologetic preamble, like, “sorry if this is a sensitive question,” or “I don’t know if you’re tired of talking about this, but…” The reality is that, for the most part, our actual sensitivity…
[115] Preregistration Prevalence
Pre-registration is the best and possibly only solution to p-hacking. Ten years ago, pre-registrations were virtually unheard of in psychology, but they have become increasingly common since then. I was curious just how common they have become, and so I collected some data. This post shares the results. The data From the Web of Science…
[114] Exhibits 3, 4, and 5
We recently presented evidence of data tampering in four retracted papers co-authored by Harvard Business School professor Francesca Gino. She is now suing the three of us (and Harvard University). Gino’s lawsuit (.htm), like many lawsuits, contains a number of Exhibits that present information relevant to the case. For example, the lawsuit contains some Exhibits…
[113] Data Litigada: Thank You (And An Update)
Thank You A few months ago we reported evidence of data tampering in four papers that have since been retracted (or re-retracted) (Colada[109] .htm). A few weeks ago we were sued for doing so (Vox .htm). Lawsuits can be very expensive, and a thoughtful and generous group of colleagues and supporters started a fundraising campaign…
[112] Data Falsificada (Part 4): "Forgetting The Words"
This is the last post in a four-part series detailing evidence of fraud in four academic papers co-authored by Harvard Business School Professor Francesca Gino. It is worth reiterating two things. First, to the best of our knowledge, none of Gino’s co-authors carried out or assisted with the data collection for the studies in this…