Memory, Misinformation, and Loftus and Palmer, Revisited
A few months ago, I floated a question in a post. Anita Eerland and I had applied for funding for a large-scale replication project. Early reviews of the proposal were very positive, but there was some concern about whether we would be able to recruit a sufficient number of labs to participate. The question, then, was simple enough: would anyone be interested in a replication of Elizabeth Loftus and Gregory Palmer’s famous “car crash experiment”?
The reviewers’ concern was quickly allayed. Responses came in from all over the world, often within minutes of one another, and before long the tentative idea had solidified into something more substantial—if not gargantuan: no fewer than seventy-five labs willing to commit their time and resources, even before there was certainty about funding!
No more uncertainty: the project has been funded, and we will be properly underway soon.
For practical reasons, project-specific communication will run through our mailing list. I will use Craving Coherence, however, to share occasional updates with the broader community—less about logistics, more about how the project is unfolding and what we are learning along the way.
At the moment, what’s “cooking” is both exciting and uneven (I realize this sounds less worrying when referring to a budding replication project than to what’s about to come out of a kitchen to your table).
The project is genuinely international, but the map still has clear blind spots. Africa and large parts of Asia remain underrepresented. We have reached out to open science communities in Africa, without much success so far. This may change. Labs can still sign up until February 1, and I hope they do. Replication gains much of its meaning precisely from diversity of context: linguistic, cultural, and institutional.
The sheer number of participating labs also opens doors that would otherwise remain closed. Limiting ourselves to a single direct replication—however careful, however well powered—would be a colossal waste of resources. With this many contributors, we can do a lot more.
Although the direct replication is the centrepiece of the project—it is, after all, what we received funding for—it also becomes feasible to explore principled variations on the original paradigm. Different tasks, different stimuli, slightly different ways of probing the same underlying phenomenon. This is something I have discussed before with Elizabeth Loftus, and it is a conversation I expect we will continue within the project. Viewed this way, replication does not have to mean sameness. It can also mean structured curiosity.
What’s more, for several languages—German, Spanish, French, Italian, Dutch—we now have 6 or more labs able to run the same experiments. This makes it possible to look beyond individual outcomes and examine patterns at the level of language groups.
All of this adds several layers of complexity to the design. We have done our best to compute appropriate sample sizes for the direct replication, but the broader project will require more sophisticated sampling plans. We are therefore very glad that Daniël Lakens has agreed to lend his extensive expertise to this part of the project.
When Brent Donnellan, Rich Lucas, Alex Etz, and I started writing Making Replication Mainstream, it was still necessary to argue that replication deserved a place at the center of the field rather than at its margins. At that time, the idea that more than seventy-five labs might coordinate around a shared question would have seemed more aspirational than practical. Looking at this project now—its scale, its international reach—it strikes me how ordinary it has begun to feel. Evidently, replication is no longer only something we talk about when things go wrong, but something we increasingly build into how we do science.
References
Loftus, E.F., & Palmer, J.C. (1974). Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning and Verbal Behavior, 13, 585-589. https://doi.org/10.1016/S0022-5371(74)80011-3.
Zwaan, R.A., Etz, A., Lucas, R.E., & Donnellan, M.B. (2018). Making replication mainstream. Behavioral and Brain Sciences, 41, E120. doi:10.1017/S0140525X17001972
Zwaan, R.A., Etz, A., Lucas, R.E., & Donnellan, M.B. (2018). Improving social and behavioral science by making replication mainstream: A response to commentaries. Behavioral and Brain Sciences, 41, E157. doi:10.1017/S0140525X18000961