Many in the mindfulness community express concern that the media has overstated the scientific evidence of the benefits of mindfulness—so much so that “misinformation and poor methodology associated with past studies of mindfulness may lead public consumers to be harmed, misled, and disappointed.” This raises serious questions about where and how mindfulness programs are offered, by whom, and in what context? When making claims about the effectiveness of any therapy or practice, it is important to make the distinction between anecdotal experience and scientific evidence. A great many mindfulness practitioners vigorously defend their lived experience of how meditation, yoga, and other practices are transformative. While entirely relevant, those anecdotes are not equivalent to scientific research, and do not constitute a scientific “evidence-base.” A strong evidence base refers to knowledge gained from high quality research where the benefits of a well-defined intervention are assessed using identical means, and are repeatedly and reliably observed across studies and with different samples of people. Because the field of mindfulness research is relatively new, very few programs have reached that milestone, but many claim to be evidence-based. The science of mindfulness holds considerable promise for creating that evidence base, however, according to this recent article’s authors, there are two fundamental obstacles: the lack of an agreed upon definition of mindfulness and methodological problems.
The Importance of Defining Mindfulness
Despite the broad coverage of the benefits of mindfulness in the media, we neither have a universal definition of it, nor a common understanding of what mindfulness practice involves. Some use the term to refer to activities that cultivate attention, awareness, or the retention of information, whereas others identify acceptance, nonjudgment, empathy or compassion as key. Practices run the gamut from meditation, to breathing exercises, to movement, to guided relaxation. This lack of a unified definition is reflected in the nine, distinctly different published questionnaires the recent study lists as being used to assess mindfulness in adults. What these questionnaires measure is often a reflection of the philosophical orientation or values of the author. This means that the way one study measures mindfulness may be vastly different from another, making it virtually impossible to build a consistent evidence base. The lack of agreement as to what constitutes a mindfulness practice, and the variety of types of practice that fall under the mindfulness umbrella pose a similar challenge. The problem here is not with the practices themselves, but the fact that mindfulness interventions lack a common thread necessary to make conclusions about their benefits as a whole. To remedy this dilemma, the paper’s authors recommend a move away from the blanket term of “mindfulness” toward more explicit language that clearly defines the practices used, and the specific mental states and functions that the practice is intended to benefit. Through the more precise use of the practices and the capacities that they cultivate, we will better understand what works, and how.
Why Mindfulness Research Methods Matter
The extent to which we can draw reliable conclusions about the impact of mindfulness also depends on the quality of the research methods. There are myriad factors that go into determining a study’s quality including whether or not the program was delivered consistently, whether a control group was used, and whether adverse events were monitored and reported. Many mindfulness studies lack this level of rigor, making their results speculative at best. Perhaps more importantly, the credibility of research rests in its ability to be reproduced to ensure that the results did not occur by chance. The outcomes of one study do not prove that a mindfulness program is beneficial. The ability to reproduce results may be particularly challenging for mindfulness researchers because of the lack of an agreed upon definition of what mindfulness is, and what a practice entails. There is also considerable variability in the number of hours of training received among mindfulness programs and therapies, leaving us with little knowledge about how much meditation is enough to achieve a desired effect. These problems are not unique to the field of mindfulness. David Vago, Research Director at the Osher Center for Integrative Medicine at Vanderbilt University and one of the authors of the article notes, “Many of the criticisms and key methodological issues the article raises are not specific to research on mindfulness—they can be made for current psychological and neuroimaging methods in general.” It is important to remember that there are exceptions. Thirty-five years of high quality studies examining the benefits of the “gold standard” mindfulness-based intervention, Mindfulness Based Stress Reduction (MBSR), show generally reliable evidence that 8 weeks of MBSR may lead to reductions in pain, anxiety and panic, and improved health outcomes. Nevertheless, reviews of all of the data find that MBSR may not consistently result in stress reduction, or relief from depression, anxiety, or prevent relapse into depression. There are a number of potential explanations for the lack of consistency in results, notably a lack of clarity about the definition of mindfulness, the use of many, dissimilar practices, poor quality research, and a reluctance to report negative, contradictory or neutral results. Historically scientists in many disciplines have shied away from publishing studies that did not yield positive findings, and mindfulness studies are not immune to that problem. An analysis of 124 published trials of mindfulness as a mental-health treatment conducted by a group of Canadian researchers showed that studies reported positive findings 60% more often than is likely statistically. What is even more troublesome, is that of the 21 mindfulness intervention trials registered with databases like ClinicalTrials.gov, 62% had no results published 30 months after they finished. This leaves the research community, the public, and the media with an unrealistic view of an intervention’s benefits.
Why Researchers are Being so Cautious
The team responsible for this recent study believe that biggest liability when “is that the benefits of mindfulness may not live up to the myth, and that inconclusive data are being used to inform clinical practice and public policy. For example, the National Institutes of Health (NIH) currently lists meditation (defined as a practice performed in a quiet location with few distractions, in a comfortable position, with a focus of attention, and an open attitude) as a potential treatment for pain, anxiety, depression, high blood pressure, gastrointestinal disorders, and smoking. The NIH also recommends that people seek conventional care in addition to adopting a meditation practice. In the realm of education, mindfulness based interventions (MBIs) have become widespread in schools across the United States, Canada, and abroad. But claims of these program’s benefits are often overstated. A 2017 comprehensive review and analysis of mindfulness-based interventions in schools found that “there is some indication that MBIs can improve cognitive and socioemotional outcomes, but no support for improvement in behavior or academic achievement.” Based on these findings, the authors of the review urge caution in the widespread adoption of MBIs and call for rigorous evaluation when schools choose to implement them. This sentiment is echoed by Dr. Kimberly Schonert-Riechl, Professor at the University of British Columbia, and a leader in the field of school-based mindfulness research. She suggests that those interested in adopting mindfulness programs in schools “scrutinize and look at the evidence,” but that “there is room for experimenting and doing pilot studies.” For researchers, she recommends that “we need to proceed cautiously, with the idea of allowing for innovation…but to always operate under the idea to do no harm, and to really do rigorous research.” To be sure, the intention of the research community in identifying the gaps between what the data tell us, and the lore promoted in the press, is not to disparage the research or the practice. The goal is to point out that “most of the scientific research conducted so far about the benefits and costs of cultivating ‘mindfulness’ (including neuroscientific evidence) has limitations in its interpretation and generalizability,” says Dr. Vago. “It’s best to maintain a cautious, critical attitude toward stated benefits in the popular press.” It can often feel as though the mandate to do no harm, the financial interests of the mindfulness industry, the need for solutions to pressing social and educational problems, and what the scientific evidence tells us are in direct opposition, with researchers urging caution and the press, the general public, and those who serve them demanding to know that their anecdotal experience is valid. Although this conflict is not insurmountable, the lack of funding for high quality research presents a challenge that may take years, if not decades to thoroughly resolve. In the meantime, we may agree that, while the scientific evidence-base may not live up to anecdotal experience, each perspective is valid and necessary to furthering our understanding of why mindfulness matters, and how best to incorporate it into our lives.
