Scientific Rigour, Standards of Transparency Could Prevent Research Challenges Down the Road
The links between Lewis Carroll’s classic 1871 novel, Through the Looking-Glass and What Alice Found There, and psychedelics are hard to ignore.
While the book never explicitly mentions psychedelics, and there’s a lack of concrete evidence that Carroll ever took them himself, the narrative of Through the Looking-Glass is steeped in psychedelic imagery. In the alternate reality Alice discovers by climbing through a mirror, flowers can speak, chess pieces come to life, and everything, including logic, is reversed.
In some ways, Alice’s looking glass is the focus of a July paper on psychedelic science published in the journal Frontiers in Psychology by Norman Farb, Rotem Petranker, and Thomas Anderson, who head the University of Toronto’s psychedelic studies research program. In the paper, Alice’s looking glass is a metaphor for the scientific studies researchers are increasingly using to gaze into the world of psychedelics itself, or into the alternate reality Alice explores.
Aptly titled “Psychedelic Research and the Need for Transparency: Polishing Alice’s Looking Glass,” the trio outline a checklist for researchers to follow, rooted in the principals of open science, to instill scientific precision and standards of transparency into psychedelic research. More specifically, they endorse the pre-registration of trials, open materials and data, constraints on generality, and replication.
“I think what we’re lacking in psychedelic (research) is rigour, … a kind of framework that teaches you how to do science properly,” Petranker tells Truffle Report in an interview. “We want to work as a community, we want to be accountable, and we want to be transparent.”
The Principles and Purpose of Open Science
Broadly, open science is a catchall term to describe the application of openness to all the parts of a research cycle, encouraging sharing and collaboration among scientists. The goal is to deconstruct barriers for sharing any kind of output, resources, methods or tools at all stages of the research process.
Open science is especially important when it comes to psychedelic research, Petranker, Anderson, and Farb write, because the field of study is undeniably exploding in popularity. Due to promising early results, frustrations with current approaches to mental health treatments, and shifting attitudes around psychedelics, researchers across the globe are turning to substances like LSD, psilocybin, MDMA and DMT to attempt to treat illnesses such as depression, anxiety, and post-traumatic stress disorder. Other researchers, including Petranker and Anderson, are leveraging psychedelics to study microdosing, creativity, openness, memory, and more.
According to the paper, following a standard psychedelic research checklist would not only create a more robust and transparent psychedelic science, but also protect scientists if their research is drawn into question by skeptical policymakers or institutional gatekeepers. Additionally, such a checklist would clearly communicate research limitations to the “already overzealous” public, private, and media stakeholders.
Why Researchers Should Pre-Register Trials
The first step toward transparency which Petranker, Farb, and Anderson endorse is the pre-registration of trials to “prevent undue commitment to career, ideological, financial, or other perverse incentives.”
Pre-registration means scientists must provide a description of all hypotheses, planned analyses, and sample sizes, along with planning and uploading a complete, date-stamped experimental design and analysis plan before human observation of data begins. “Pre-registering requires researchers to solve problems in the planning phase rather than only discovering issues after data has been collected,” the paper reads. “Pre-registration also protects the researcher and the broader scientific community from reliance on improperly liberal thresholds for statistical significance, and from hypothesizing after the results are known, … which directly contribute to contemporary replicability issues.”
Petranker says there are “a lot of true believers” who already know psychedelics work, and want to prove that they do. Private equity could be playing a similar game, standing to make massive amounts of money if they can prove their products work. “That’s not how science (works),” says Petranker. “In science, we want to see if something works, and if it doesn’t, (ask) why?”
The Need for Open Materials and Data
Open materials and data are also key to building a transparent psychedelic research enterprise, Petranker, Farb, and Anderson write in their paper, so “scientists can evaluate, reproduce, replicate, and explore the methods and data used to determine research findings.”
Open materials covers all questionnaires, stimuli and tasks used in the methods section of a paper, while open data asks researchers to release a complete, de-identified set of all data collected as part of a study. Adopting this approach means analyses can be rerun to check for errors. At the same time, new analyses can be attempted, and new hypotheses can emerge, encouraging collaboration and follow-up investigations.
Researchers Should Include Constraints on Generality
Petranker, Farb, and Anderson also encourage researchers to include a ‘constraints on generality’ section in their papers, which would cover the limitations of the published work. These sections should identify qualities of participants, materials, procedures and context that researchers deem necessary for observing reported effects, and also identify irrelevant qualities.
Constraints on generality statements remind researchers “to identify qualities of the messy reality of research that may play a role in limiting what we can learn from a study,” the paper reads. “Psychedelics show great promise, but we must remember that most contemporary lab studies have small sample sizes and that larger survey studies, including our own, are necessarily limited by lack of experimental control.”
Preventing a Replication Crisis in Psychedelic Research
A clear example of the need for open science, both in psychedelics and any field of study, is what’s called the replication crisis. In science, replication means if two scientists are using the same research design to study the same subject, the pair should get identical or very similar results. In some areas of study within psychology, biology, and medicine, the idea of replication was assumed, but rarely tested. Petranker chalks this up in part to a “publish or perish culture,” where researchers feel pressured to produce new and positive findings for hungry journals.
When researchers began going back to replicate integral studies, as was the case in psychology, they found a surprising amount didn’t replicate. For example, one team of psychologists working to replicate classic psychological experiments revealed in 2018 that they only succeeded in half — or 14 out of 28 — cases. Petranker says he doesn’t want to see psychedelic research facing a similar crisis, especially while the field is largely still working through its foundational research.
“Replication ensures that foundational scientific research is solid, and provides an opportunity to work together as a community of researchers,” the paper reads. “If results fail to replicate, we should regard them as less reliable, and consider looking for smaller effect sizes or using higher-power designs in the future.”
Over 130 Psychedelic Experts Have Also Backed Open Science
Petranker, Anderson, and Farb aren’t the only people in the community who are advocating for the need for open science in psychedelic research: over a hundred other experts, and dozens of organizations and institutions are on board as well.
In December 2017, Robert Jesse led the drafting of the “Statement on Open Science and Open Praxis with Psilocybin, MDMA, and Similar Substances.” Jesse is convenor of the Council on Spiritual Practices and played an instrumental role in the formation of the psilocybin research team at John Hopkins University, where he has co-authored a number of influential papers that are largely credited with sparking the current psychedelic research resurgence.
The statement is now supported by more than 130 scientists, scholars, and practitioners, ranging from experts at New York University and Imperial College London to the British Columbia Centre on Substance Use and the Czech Republic’s National Institute of Mental Health. Thirty research and service organizations also backed the statement, along with over a dozen psychedelic societies and integration circles. More than twenty supporters, organizers, publishers, conferences, philanthropists, and philanthropic organizations signed on as well.
Toward a Collaborative, Transparent Future in Psychedelic Research
“From generations of practitioners and researchers before us, we have received knowledge about these substances, their risks, and ways to use them constructively,” the statement reads. “In turn, we accept the call to use that knowledge for the common good and to share freely whatever related knowledge we may discover or develop.”
Along with endorsing open science and open praxis, the statement also backs intellectual and scientific integrity, placing common good above private gain, and striving to make expertise, findings and services available to all who might benefit from them.
While Petranker is excited by all the new research focused on psychedelics, “I see some researchers that are playing really fast and loose, and it’s scaring me,” he says. “This is a real danger, because it only takes one person to cause a negative, serious adverse effect.”
That’s where the Canadian Centre for Psychedelic Science, a research and consulting company co-founded and run by Petranker and Anderson, could come in. Five or ten years down the road, Petranker says, the hope is for the centre to be a beacon for open science in psychedelics. “People will come to us when they want to assess whether a certain program of research is rigorous, and whether it complies with the principles of open science,” he says. “I would like for our impact on discipline to be helping to implement open science.”
