In a pre-COVID-19 world, this week would have seen evidence-based medicine experts gather in Toronto for EBM Live 2020. However, like many other meetings during the pandemic, it has been postponed. Here, I reflect on last year’s meeting and pharma’s journey in improving the reporting of clinical trials.

In July 2019, expert researchers, clinicians and patients gathered in Oxford to discuss the current state of evidence-based medicine (EBM) and set directions for the future. As in previous years, I joined them. As one of the few pharma representatives at EBM Live, I was privileged to have an outsider’s view, and it gives me no pleasure to report that EBM feels to be running out of steam. The EBM movement has indeed transformed healthcare but, in some ways, is now a victim of its own success. Despite efforts to move on, many in the field are still fighting the same old battles. So, what does the picture really look like for pharma today? This blog post sets out my personal opinions.

In little over 25 years since the concept came to prominence, EBM has changed the way in which medicine is practised around the world. I have enormous respect for the achievements of the movement; for example, systematic reviews, meta-analyses and indirect treatment comparisons are now mainstays of decision-making about the availability and use of new medicines. As a sometime systematic reviewer myself, I know the excitement of seeing a clear pattern emerging from complex data and the joy of using a rigorous methodology to reveal something that no one has seen before.

Much of the programme at EBM Live was devoted to calling out bad behaviour within pharma, as in previous years. These stories, while shocking, are old. In fact, even the examples I hadn’t heard before were never less than 10 years old. I understand the sense of betrayal felt by EBM practitioners who believe their hard work was subverted by unscrupulous behaviour within certain pharma companies. That said, now is the time to move on.

Pharma has cleaned up its act beyond all recognition. At Oxford PharmaGenesis, we have sought to bring perceptions about pharma up to date. We have attended EBM Live (formerly Evidence Live) for 4 years, contributing to presentations, workshops and informal discussions. In 2016, we presented data showing that professional medical writing support is associated with more timely and complete reporting of clinical trials. In 2018, we presented an analysis demonstrating that pharma companies have transformed their disclosure performance, with higher rates of disclosure observed for pharma (74%) than for non-pharma sponsors (46%). Indeed, Ben Goldacre and colleagues from the Centre for EBM at the University of Oxford are showing that pharma companies are commonly reporting 100% of their clinical trials, with no evidence that they are driving outcome switching or other questionable practices. As Ben told me with characteristic circumspection, “Nobody is doing well. Nobody is meeting patients’ expectations. But pharma is currently performing better than academia in many quarters, and this should be celebrated. We must capitalize on it.”

EBM practitioners should now embrace initiatives that seek to make progress in the field. I was glad to be able to provide input to two such initiatives at previous EBM Live workshops, namely the Declaration to Improve Biomedical & Health Research and the EBM Manifesto 2.0. Such initiatives are advancing of the field of EBM by encouraging engagement between diverse stakeholders and should include the perspectives of pharma companies, which generate so much of the evidence on which EBM depends. Sustained scrutiny of pharma companies by experts in the EBM field was transformative. It played a crucial role in the transformation of pharma’s disclosure performance, leading to billion-dollar fines and sweeping policy changes. And yet, the repetition of old war stories about pharma dating back 15 years or more contributes to a feeling of stasis in the field – that nothing is ever going to change. Although many of the problems that EBM Live raises will, to some extent, always be with us, reporting of trials by pharma is an area where the field can be justly proud of its achievement. If EBM practitioners take time to celebrate this, then they will learn lessons for driving future change and, most importantly, remain motivated and relevant in the decades to come.

2 thoughts on “Evidence-based medicine no more?

  1. Evidence-based medicine or Vive Le Communisme.
    Polemic Notes
    Vladimir Zaitsev

    Introduction
    Many theories, recommendations and stipulations are useful and sound at their core – take, for example, added value and its distribution – up until the point they become an ideology.

    In 1980, David Sackett, Brian Haynes, Gordon Guyatt and Peter Tugwell, then-young researchers at the McMaster University in Canada specializing in mathematical statistics and probability theory, came up with the RCT principles, laying the groundwork for evidence-based medicine (ЕВМ). In Russia, it is more commonly known as evidentiary medicine (EM). Statistical data processing as such was nothing new for medicine, but these Canadian mathematicians designed a specific pattern for this kind of statistical research.
    RCTs are conducted in accordance with several basic principles. First and foremost, it’s randomization. Secondly, the controlled trials should ideally be “blind” or even “double-blind” and “triple-blind” so as to prevent researchers’ bias from influencing the outcome.
    As for randomization (random allocation of patients taking part in the study), in statistics it’s used in case there are unknown or unaccounted for variables or factors that could affect the result, not just the obvious parameters such as the type of disease and its severity, other medical conditions, age and gender. And what comes to the fore here is one of the fundamental laws of probability theory – the law of large numbers. It is frequently invoked by mathematicians who criticize EBM [6]: with small numbers, randomization is pointless.
    Yet in Europe many researchers, use randomization even with relatively small data samples, since a number of medical journals are reluctant to publish articles without this magic word.
    Statistical data processing as such is a necessary and useful tool applied in almost every scientific field out there.
    But only in medicine have such studies of average probability turned into dogmatic proof of anything and everything.
    It is not EBM that is harmful as such, but this approach that allows its principles not just to dominate the minds of passionate EM disciples, but to usurp them. And in real life, pathophysiological criteria are increasingly driven out by probabilistic approaches.
    At specialized medical forums, we hear more and more often calls to introduce a mandatory mathematical statistics course to medical universities, and a number universities have already set up these departments. Rather than demonstrating expertise in pathogenesis and etiology, doctors, especially young ones, boast about knowing the difference between statistical methods proposed by Student and Fisher.
    That said, a number of experts, even those working at organizations and facilities whose names have “evidence-based medicine” in them, are very critical if not downright sarcastic about this blind EBM worship [1-7].
    Back in 2002, one of the most highly respected scientific medical journals in the world, the British Medical Journal (BMJ), published a satirical report about EBM [1]. The authors called EBM a full-blown religious movement, complete with inquisitors for those who dare defy its commandments, i.e. refuse to treat patients only in accordance with the EBM cookbook. Like any other religion, EBM proselytizes aggressively, recruiting new members at various seminars and colloquiums, as well as via guidebooks and other publications.
    The main argument of other articles critical of RCTs is that RCT results becoming the dogma and ultimate proof in modern medicine hinders the development of pathogenic research methods.
    For example, the article Does evidence based medicine do more good than harm? [2] focuses on just that. The leading role of EBM in judging the effectiveness of various drugs sidelines all the other methods and gets in the way of real proof. More importantly, it does not foster critical thinking in doctors. The author – a professor of clinical epidemiology, no less – concludes that all things considered, EBM could be doing more harm than good.
    This kind of ideology mainly benefits the pharmaceutical industry, which is where the lion’s share of all medical money is.
    Only big pharmaceutical companies can afford the large-scale and extremely costly RCTs, which are a prerequisite for any drug or treatment to be included in the standards. And then there is the constant brainwashing, when doctors are told that medicines or treatments untested via RCTs “have not been proven” effective. The competition in the pharmaceutical market is cut-throat as it is.
    A few words on standards. Unlike recommendations, which were quite common in the past, standards is the kind of normative document that you cannot deviate from, because you could be risking a criminal charge.
    That’s the reason for the many cases when the results of clinical trials were tweaked or even outright doctored: not getting in the standards means losing billions.
    One of the works, peculiarly titled Evidence-based medicine was bound to fail [7], claims that the hard-selling of EBM principles serves the interests of big pharma and that despite a certain positive contribution EBM has, we need to recognize its limitations. Determining average possibility is not a scientific approach, and we should work harder on improving the pathogenesis-centered approach to research and treatment.
    Even Dr. Vasily Vlasov, the main advocate of EBM in Russia and president of the Russian Society for EBM for 10 years, touched on that issue in his article Evidentiary Medicine As A Drug Promotion Device [8].
    There are a lot of critical articles on the pharmaceutical industry. Just take Ben Goldacre’s Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients (it was translated into Russian) [9]. A British doctor and researcher, Ben Goldacre used to work at the Centre for Evidence-Based Medicine at the University of Oxford.
    For the pharma industry though, all this criticism is nothing but flea bites.
    Business is business, and he who pays the piper calls the tune.
    Fun fact: in the US, large pharma companies spend more money on lobbying than even the arms manufacturers.
    But let’s get back on track.
    Even possible rigging aside, what conclusions can be drawn from RCTs with their average probability? That the probability of drug A being more effective across a large group of patients is N percent higher than that of drug B (or a placebo). But that doesn’t mean that this is the best or even indicated option for a particular patient.
    If large-scale statistical research shows that blondes have a higher chance of getting married and that there are fewer single women among them, it does not mean that all women should dye their hair and that each of them would look better with it.
    However, this approach to “proving” whether a drug is effective or not has become so widely accepted and legitimized that sometimes it results in completely counterintuitive studies. For example, extensive averaged-probability research (with RCTs) was conducted to determine the antibiotic resistance of pathogenic bacteria strains, chlamydia in particular, even though a much easier and more reliable way to do it would be to opt for microbiological studies, both in vitro and in vivo.
    Or take the respectable AstraZeneca, a large manufacturer of statins, which spent dozens of millions of dollars on an extensive fully EBM-compliant epidemiological study involving 17,802 mostly healthy men and women without any signs of hypercholesterolemia (!) taking statins over a long period of time [10]. They said it was for preventing cardiovascular events and used C-reactive protein as the key indicator, which in this case is completely meaningless, as elevated C-reactive protein level can simply be a sign of any inflammation, like with an ingrown toenail. There is hardly a marker more convenient to get the desired result and increase sales.
    The EBM era saw a sharp increase in large-scale epidemiological studies, a number of which benefited only the researchers’ bank accounts.
    For example, there have been so many epidemiological studies on the dangers of excess salt consumption. And suddenly there came a long-term, extensive American-Israeli epidemiological study involving more than 8,000 people that claimed the exact opposite [11]: that mortality rate rises when salt consumption is too low, not too high.
    In fact, for the overwhelming majority of people it makes no difference whether they indulge in pickled herring or pickled cucumbers or not. With a normally functioning water and salt homeostasis, the secretion of aldosterone regulated according to the body’s needs and so on, the body is perfectly capable of maintaining the necessary concentration of sodium ions. The solubility of sodium chloride is such that with urine it can be excreted in larger quantities than a person is capable of consuming. And only a small number of people (whose water-salt balance is off) have to regulate their sodium chloride intake.

    Conclusion
    The future lies with the real medical science and the pathophysiological approach, with all its methods, cause-and-effect links and the correlation between clinical trials and corresponding surrogate indicators. This is the future we have to work on advancing instead of denying it. But when doctors get brainwashed with average probability, when one of the main signs of a doctor’s thinking process is “the ability to critically assess and determine the correct use of randomization and statistical methods,” that future moves one step farther away from us.  
    Notes
    1. EBM: unmasking the ugly truth
    Clinicians for the Restoration of Autonomous Practice (CRAP) Writing Group

    BMJ . 2002 Dec 21; 325(7378): 1496–1498.

    2. M G Myriam Hunink (professor of clinical epidemiology and radiology)
    Does evidence based medicine do more good than harm?
    BMJ 2004; 329
    Reviews
    Evidence-Based Investigation into the Relation Between Sexual Intercourse and Pregnancy
    Jacob M. Puliyel, Noopur Baijal, Dherain Narula. (10 November 2004)

    3 Medicine, Health Care and Philosophy
    August 2005, Volume 8, Issue 2, pp 255–260
    The challenges of evidence-based medicine: A philosophical perspective
    4. S Doherty
    Evidence‐based medicine: Arguments for and against
    Emergency Medicine Australasia, 2005 – Wiley Online Library
    5. Clifford G. Miller BSc ARCSa and Donald W. Miller, Jr., MDb
    a Solicitor, Supreme Court of England & Wales and former Lecturer in Law, Imperial College, London, UK
    b Professor of Surgery, Division of Cardiothoracic Surgery, University of Washington School of Medicine,
    Seattle, Washington, USA.
    The Real World Failure of Evidence-Based Medicine
    The International Journal of Person Centered Medicine
    Volume 1 Issue 2 pp 295-300 June 9, 2011

    6. D. Stephen Hickey BA PhD MSB CBiola, Andrew Hickey Dip Comp (Oxon)b and Leonardo A. Noriega BA MSc PhD LLB(CPE) MBCSc
    a Head of Newlyn Research Group, Newlyn, Penzance, UK
    b Senior Researcher, Newlyn Research Group, Newlyn, Penzance, UK
    c Senior Lecturer, Faculty of Computing, Engineering and Technology, The Octagon Staffordshire University, Beaconside,
    Stafford, UK

    The failure of evidence-based medicine?

    European Journal for Person Centered Healthcare Vol 1 No 1 pp 69-79 (2013)

    7. Fava GA
    J Clin Epidemiol. 2017 Apr;84: 3-7.
    Evidence-based medicine was bound to fail: a report to Alvan Feinstein.
    8. В.В. Власов
    Доказательная медицина как средство продвижения лекарственных средств.
    “Ремедиум”, N 4, апрель 2007 г.
    9. Ben Goldacre
    Bad Pharma: How Drug Companies Mislead Doctors and Harm Patients
    «Faber and Faber» USA 2013
    10. Paul M Ridker, M.D., Eleanor Danielson et al.
    Rosuvastatin to Prevent Vascular Events in Men and Women with Elevated C-Reactive Protein
    N Engl J Med 2008; 359:2195-2207

    11. Hillel W. Cohen, Susan M. Hailpern, Michael H. Alderman, Sodium Intake and Mortality Follow-Up in the Third National Health and Nutrition Examination Survey (NHANES III). Journal of General Internal. 2008; 23(9): 1297–1302. doi: 10.1007/s11606-008-0645-6

Leave a Reply