Cory J. Clark, Bo M. Winegard, Jordan Beardslee, Roy F. Baumeister, and Azim F. Shariff had a paper published in Psychological Science - a/the top journal in psychology. Only months later, now in June, they have decided to retract it.
The retraction notice talks about data based on small and skewed samples, and bad measures that opens up for questionable research practices. Thus, pretty standard bad science in psychology, in particular the small samples and bad measures. See e.g. Malte Elson’s site Flexible Measures on how aggression is measured. I have myself talked about the bad measures used in investigative psychology in an episode of Everything Hertz (about 20 min into the episode). Low level of generalisibility is also very common in psychology.
Although I agree we should expect a higher standard from psychology’s flagship journal, I was very surprised to see a retraction based on these reasons only. A retroactive disclosure statement seem to have been the more obvious option. As we write in our manuscript currently under re-review at Psych Science:
“[I]t is not at all clear that widespread adoption of retractions would be an effective, fair, or appropriate approach. Willén (2018) argued that retraction of articles in which questionable practices were employed could deter researchers from being honest about their past actions. Furthermore, retracting papers because of questionable research practices (QRPs) known to be widespread (e.g., John et al., 2012) could have the unintended side effect that some researchers might naively conclude that a lack of a retraction implies a lack of QRPs. Hence, Willén (2018) suggested that all articles should be supplemented by transparent retroactive disclosure statements. In this manner, the historical research record remains intact, as information would be added rather than removed.”
But then I read the Editorial and realised that the retraction may not really be about the methods and data after all.
The journal’s Editor in Chief, Patricia J. Bauer, wrote an Editorial on the retraction, and this Editorial does not (only) discuss how the journal want to handle the generally low standard in psychology, the replicability crisis or how the reviewers should be instructed to handle it. Instead is the Editorial focusing on a completely different reason for retraction - something that is not mentioned at all in the retraction notice - namely, racism.
"As social scientists, we have a responsibility to be sensitive to the political, social, and cultural issues raised by our work. […] We must be especially sensitive when the topics with which we are dealing are associated with a history of injustice and when the message of our work could be inflammatory or incendiary.
In the case of the now-retracted article, some readers may debate whether the authors themselves were sufficiently sensitive to these issues. It is not my place to voice a perspective on that concern. It is my place to take a stand on whether in our handling of the manuscript, Psychological Science was sufficiently sensitive. I have concluded that we were not. We failed to recognize that the message of this article could be interpreted to have racial overtones and thus could be highly controversial. We therefore failed to act to mitigate the potential harm to which the message could contribute."
“And because words matter, we also will be paying closer attention that in the articles we select for publication […] that conclusions and their possible implications are conveyed in a socially sensitive and scientifically responsible manner. These actions will make both our journal and our science more socially responsible. […]”
“I close with an apology to the field and the broader society for any harm to which we contributed by publishing research without sufficient sensitivity.”
The Editor emphasises:
“We should not and will not shy away from publishing articles on sensitive political, social, and cultural issues. But what we must and will do is exercise greater care in our handling of all submissions, including those on sensitive topics.”
She writes that the journal will add a submission type called “Further reflections” to supplement papers published on controversial topics. A more important solution would be to require preregistration, open data and materials, and that the article is published open access. That is, no controversial topics or claims are published without preregistration and complete public access to data, materials, and paper. I’m surprised that nothing like that was mentioned at all in the Editorial and neither have I seen anything about this in the few discussions I’ve seen on Twitter. These solutions would have changed things also with Bem’s 2011 publication in Psychological Science (see Wagenmakers et al, 2011, for an overview of the issue), unlike “Further Reflections” comments.
Correct me if I’m wrong but a retraction due to methodological issues in this case seems to be outright wrong.
Link to Psychological Science where both the retraction notice and the editorial are found: https://www.psychologicalscience.org/publications/psychological_science/clark-2020-retraction-editorial