Sunday, 8 April 2018
Evidence-based medicine viewed through binoculars? - by Robin Daly
Listen to any news broadcast and you’ll be almost guaranteed to hear the word ‘evidence’ - usually several times. It’s the buzzword of our times and is used to try to establish credibility and trustworthiness in all walks of life.
And this is important, of course, All too often, vast amounts of effort, time and money have been wasted on initiatives for which there is no sound basis. Unfortunately, evidence is wildly variable, and often doesn’t fit the bill as a ‘sound basis’. This is notably the case in medicine, an area where, in a relatively short period, evidence has become king. A recent conference I attended, at which the word ‘evidence’ must have been up there in the top ten of all words used, prompted me to think about the nature of evidence and what really amounts to trustworthy evidence. The train of thought that resulted in this piece was prompted by a representative of NICE who said words to the effect of ‘… even including some data from patients’, as a sort of afterthought.
Our healthcare service currently has two policies that are both seen as central, but work in direct opposition to each other: evidence-based medicine (EBM) and patient choice.
EBM has inevitably become steadily more proscriptive. Despite its promising beginnings as an attempt to bring some rational basis to a largely unscientific medical system, and even though its original intention was to support clinical decision-making, in practice, medicine has become increasingly ‘cookbook’, and doctors who stray too far or often from the recipe are laying themselves open to harsh criticism or even disciplinary scrutiny.
Patient choice, on the other hand, seeks to make medicine less proscriptive. Patients often have very varied views of what constitutes good medicine, and so the spectre rears its head of a patient making a choice that fits well with their personal philosophy of healthcare, and that they would like for their own physical and mental wellbeing, but that is not considered good EBM.
This institutionalised schizophrenia has largely been overlooked, but the result is plain: EBM is being enormously successful as it is a natural extension of the ‘doctor as god’ setup that we are all too familiar with. Patient choice however, despite being on government agendas since the 1980s, and despite being enshrined in the NHS Constitution as the fourth principle that guides the NHS, has made virtually no headway at all in 40 years.
How are we to make any progress with patient choice? This single issue threatens to condemn British citizens to an antiquated model of care that has more in common with the 1950s than with virtually every other aspect of our modern, consumer-centred world. The answer could lie in a re-examination of what constitutes good evidence.
As a modern consumer, I have a range of choices whenever I want to choose goods or services:
1. I can read the manufacturer or supplier’s sales literature
2. I can consult an ‘expert’ review
3. I can look at the ratings of previous customers
These of course, are all in addition to going somewhere to look at a product or talk to a supplier. But assuming, for the sake of argument, that we are buying over the internet, then we have the three forms of ‘evidence’ above to inform our choices.
Well clearly, the manufacturer / supplier is likely to do a thorough job on explaining the advantages and strengths of their offering. We may enjoy reading the sexy brochure about that fabulous new car, but it will probably tell us relatively little about the real-world experience of owning it. So, its usefulness is limited to some facts and figures.
In contrast, we traditionally turn to ‘independent experts’ for an unbiased and in-depth view of which car really is the most reliable or economic, which computer is the fastest, and so on. Experts certainly have their place and can take some of the grunt-work out of narrowing down options. But all too often, scrutiny reveals that the ‘independent’ tag is not as watertight as we might have expected. The motivation for huge businesses to want to influence experts to be more favourable than they might otherwise be, is an obvious one, and can often result in our trusty experts being exposed to practically irresistible levels of inducement. Even if not directly influenced in this way, a ‘reviewer’ is likely to constitute a one-man-band, incorporating a host of opinions, hobby-horses, blind spots and general subjective thinking. For this to have no impact on the resulting reviews is implausible. A team of reviewers may seem a route to more objective thinking, but the power of ‘group-think’ is too well documented to be ignored.
The third option is an interesting and recent one. For certain, many of the reviewers are similarly afflicted by biased subjectivity in the way that an expert can be. But these are all people with their own real-world experience of the goods or services in question, and they’re not subject to the same levels of group-think that a team can be. If there are three reviews and one of them is a rave 5-star number, and one a complete 1-star bin job, it tells me very little. But if, as happened to me when choosing some binoculars recently - my first pair ever, and a subject I know nothing about - more than 1,200 reviewers give an average of 4.8 stars and they are the top-selling model, then that’s all I need to know.
So, what we have here is a hierarchy of evidence. EBM has its own hierarchy of evidence, but it’s quite clearly rather different.
In my consumer (read ‘patient-centred’) example -
• No 1 is what I shall call 1950’s evidence. Its what people got back then - they probably thought it was okay, and anyway it was all there was
• Then came along late 20th century consumerism and the rise of No 2, the ‘independent expert’. This seemed fantastic at the time, and some people still ‘believe’ in them today
• But the 21st century is definitely the territory of No 3, the ‘consumer rating’
From my perspective, No1 is practically useless and always to be viewed with caution; No 2 can be useful, particularly when No 3 is lacking, but No 3 is the no-brainer, immediate, go-to type of evidence.
Now let’s examine the hierarchy of EBM. Right at the top, the ‘gold-standard’ of EBM is the randomised controlled trial. These are, in the main, the product of the companies that manufacture the product or deliver the service. Therefore, in my ranking as a consumer (read patient) this evidence is extremely poor.
EBM seeks to counteract some of the epic shenanigans of the pharmaceutical industry by forming bodies to regulate what choices are made by the NHS, and not to simply take the industry’s word that their products are fantastic. Well this is definitely a step in the right direction, but as in the consumer world, these ‘independent experts’ are subject to all the same forces as their consumer equivalent. Investigations have regularly exposed the levels to which the members of such regulatory bodies (and ‘trustworthy' charities) are enticed to be less than objective, how a ‘revolving-door’ exists between powerful positions in industry and in regulation (and charities) and the stark level of ‘group-think’ imposed on members. One that comes to mind, but is so accepted and so huge as to be mistaken for the very ground on which healthcare stands, is that to be even considered as a medicine capable of making people well, it’s pretty much the rule that a substance must be a patentable, new-to-nature toxin - in other words a product of the pharmaceutical industry. Tragically, all too often the regulators are clearly seen to be defending the interests of industry, rather than of the public.
As for evidence from patients, this is considered the very lowest form - merely ‘anecdotal’- so much so that there is virtually no interest in even collecting it. Just imagine the scenario - the block-busting drug with the incredible pharma-funded trial results, given the stamp of approval by the pharma-supported regulatory body gets a 1-star rating from users who say they don’t feel any better and the side-effects are destroying their quality of life. Who will benefit from that ‘consumer’ knowledge? Why consumers of course. And what threat does such information pose? The failure to get away with a decent ‘honeymoon period’ of massive profits before everyone realises they’ve been had.
The point I’ve been trying to illustrate is that at present EBM is in direct conflict with patient choice for one simple reason: the hierarchies of industry and patient evidence are completely opposite. From where I stand as a patient / consumer, the most trusted evidence is the least trustworthy, the least trusted, the most reliable. Is it any wonder that healthcare is failing its patients?
In one deft move, we could overcome this tragic mismatch. Healthcare could step into line with the rest of the deliverers of services and products, by changing EBM such that what patients, consumers of healthcare, have to say about a medicine or procedure is the MOST important piece of evidence - not an afterthought, or nice-to-have, feel-good extra, but one that the NHS pursues voraciously, in the interest of delivering the very best to the most patients. This way they would be living up to their constitution. Patients would benefit from knowing and receiving the most effective choices; the NHS would benefit by increasing patient satisfaction and outcomes and thereby reducing costs; and even the apparent casualty, business, would benefit from a level playing field in which aims shift away from the almost exclusive focus on the bottom line, towards a balance between profits and patient-endorsed effectiveness.
At a single stroke (okay, maybe a bit poetic for a complete inversion of an international system!) EBM and patient choice are working together, pushing in the same direction, building better healthcare for us all.
 I am ignoring the potential for dodgy dealings with consumer rating results. Although possible, it would be commercial death for any business if discovered