STAT: My Daughter’s MS Diagnosis and the Question My Doctors Couldn’t Answer

Is there a dietary treatment for multiple sclerosis? And if so, why is the medical establishment ignoring published academic research that started in the 1950s proving it?

Maria Bustillos | Longreads | September 2016 | 40 minutes (10,049 words)

 

I.

In the first days of 2014, in her senior year at Oberlin and just a few days before the winter term she’d arranged to spend in France, my daughter Carmen’s legs went numb. First her feet got all tingly, then her ankles, calves, and knees. Over three days or so, the numbness crept up to the base of her rib cage, and then stopped. But it didn’t go away—a weird sensation all in her skin, almost as if the whole lower half of her body had been anesthetized. Shingles, the internist told us—really?—okay. The acupuncturist, too, told us he’d been seeing anomalous cases of shingles cropping up in younger people. Carmen seemed to get a little better, and off she went to Paris; the tingling and numbness subsided slowly over the next several weeks, just as we’d been told they would, and the episode faded from memory. But about a year later, they came back again: Not shingles, after all.

Carmen in a hospital bed, uncharacteristically quiet and gloomy, the dark jungle of her curls against slick, plasticky polyester pillowcases. IV steroids, and more and more tests. Legs pretty numb, still. From pregnancy onward, I imagine, most parents harbor a cold little drop of inward fear, even as each day passes peaceful and undisturbed, through birth and babyhood and all the playdates and sleepovers and math tests, rock shows and summer vacations; at any moment, perhaps, from out of nowhere, comes the pounce. Here it is, then. Multiple sclerosis: I didn’t know anything about it really, beyond calamity, wheelchairs, and Annette Funicello. Instant by instant I composed my face and steeled myself as best I could for… what?

For every cliché in the world, naturally. A soul-wracked family, just like the ones you’ll see every day on the Lifetime Channel and the evening news; a brave young person, scared and in trouble; you register a fleeting hope that things will work out for them, in fact or fiction, as you flick to the next station. Now it’s your turn, but you won’t be changing the channel. Can this thing be treated? What is it? How do I discover how bad this will get? Or maybe let me just jump out this motherfucking window this minute, because I’m going to die of the panic alone.  

* * *

Carmen, on a post-undergraduate Grand Tour in Europe.

Carmen, on a post-undergraduate Grand Tour in Europe.

By chance a leading professor in the neurology department of this Top Hospital was on rounds and came to see my daughter: an Iron Doctor whose medicine very clearly reigned supreme. A shoal of residents bobbed around this handsome, winning, authoritative man. A delightful guy, 60 or so, with a youthful, open countenance, perfect manners and brilliant blue eyes. I couldn’t wait to talk with him. It was my first chance to speak with someone who’d treated people with MS.

He made a practiced speech to my daughter, my ex, and me. MS is an as-yet-incurable disorder in which the immune system mysteriously attacks the protective myelin sheath around the nerves in the spine and brain, causing lesions that in most cases will eventually result in some degree of permanent damage to the central nervous system. We can offer treatments, he continued smoothly, and there is hopeful research. With care, there’s every chance Carmen can live a long, active life. But MS is generally understood to be a degenerative neurological disorder.

What care, though, exactly? What is care. It turned out that by “care,” he meant drugs.

Almost all our brief talk revolved around medications that could halt the progression of the disease; he sketched out the options, some injectable, others in pill form. The complicated names swam past me: Interferons, natalizumab, glatiramer acetate. He is of the old school, he told us, and prefers the injectable medications—Carmen would inject herself every day, or every other day. Many of these drugs have been around for decades. There’s been less experience with the new pills, which come with significant side effects, though some patients tolerate them well.

Some of this we’d already learned online. We’d also learned that some small number, perhaps five or ten percent, of those diagnosed with MS will have a “benign” version of the condition, with a few flare-ups that go into remission, leaving no permanent damage. But of those who are diagnosed and go untreated, one-third will wind up permanently disabled within 20 years’ time. Which third, nobody can say. Some people sink straight into an irrevocable decline after their first attack (“flares,” these inflammatory attacks are called, or “exacerbations”), with pain and nerve damage growing so severe that walking or even standing unaided becomes impossible within only a few years, or even months; others have a few twinges, heal up entirely and never get sick again—and there’s everything in between. MS is like Russian roulette for your body, for your whole life.

What books do you recommend, I asked the Iron Doctor, as crisply and collectedly as I could, given that my kid was in a hospital bed and the floor was lurching uncontrollably around. What websites should I read? What must I do? The National MS Society has a website, he said. He recommended not one single book. (His unsolicited visit, which was billed separately from the hospital bill of around $30,000, cost $693.) Okay… what else? Are there lifestyle changes indicated, exercise or dietary recommendations?

He said no, no there aren’t.

No! There is nothing you can do or read! There is nothing you can learn, nobody to talk to. Visit this website—if you must—since you ask. Two and a half million people around the world are estimated to have multiple sclerosis; we can safely assume that a large percentage of them are leaving their doctors feeling as lost as we were that day. Immediately we set out to read everything we could lay our hands on, starting with the National MS Society website.

What you will mostly learn via the FAQs of the National MS Society website is that there are many, many medications available for you. Medications for Modifying the Disease Course, for Exacerbations, for various symptoms such as Bladder Problems, Dizziness and Spasticity. There are some community resources, but you have to work to find them, and the offerings are minimal; there’s a blog with new posts about once per week, and a few thinly-trafficked but good discussion boards. There’s a chat room for the newly diagnosed that contained, day after day that I checked, just me.

Elsewhere, the National MS Society website recommends a “Comprehensive approach” made up of several “parts that make up the whole” of a treatment plan. Ha! Pretty much the only comprehensive thing about this approach is that it comprehensively responds to every symptom with another pill. We have “Modifying the disease course,” (with any of 12 approved meds); “Treating exacerbations” (with meds); “Managing symptoms” (with literally 60 different meds, plus maybe a physical therapist will come and help you learn to use a cane). And finally, there is “Emotional support”—did you guess? Meds! You might need antidepressants and/or anti-anxiety meds, and/or counseling, possibly, to deal with your anxiety, mood swings and depression.

The National MS Society does give a brief nod to what it calls CAM (Complementary and Alternative Medicines).

screen-shot-2016-09-13-at-8-38-51-pm

So, “no diet has been proven to modify the course of MS,” and acupuncture is “finding its way into Western medicine” as if it were some kind of infestation. Again, this website was the sole source of information recommended to us by a senior neurologist at one of the most renowned teaching hospitals in the United States.

Elsewhere online we found some studies and treatments that sounded promising, and countless others evoking pure Grand Guignol. There are practitioners who will sting you with live bees, pop you into a hyperbaric chamber, or open up the veins in your chest and neck with stents in order to “improve the drainage” of spinal blood. There’s a special type of acupuncture where they spin the little needles on your scalp, somehow, when they prong you. There are true believers in all these things, many of them angrily declaring that the Medical Establishment is Concealing the Truth.

Nearly every day since this adventure began I have had occasion to recall the film Man on the Moon, in which Jim Carrey, in the role of comedian and cancer sufferer Andy Kaufman, is made to confront the theater at the heart of medical charlatanism; where ritual, wishful thinking, performance, divination and medicine combine in a Kafkaesque brew of cruelty and hilarity.

In time we found many invaluable resources: databases, research blogs, the Public Library of Science, PubMed, conference presentations and videos, the government’s clinical trials website, and, best of all, we found forums and groups of people with MS sharing their stories, not just about the success or otherwise of various treatments, but about their lives, their work and relationships.

The good news is that we started to learn about therapies Carmen might safely try in order to stay healthy. The bad news is that I came to the eventual conclusion that the medical establishment is effectively harming people with MS—in a number of ways, but most especially by failing to inform them of a very simple, low-cost, vegan-plus-fish dietary regimen, based in classic research, that might help them stay well. Vegan plus fish—that is it, pretty much. There are additional valuable details to learn about, but basically it’s a three-word prescription. Animal fats appear to play a significant role in making MS patients sicker; fish oils help keep them healthier.

Dr. Roy Swank. Photo via Swank MS Foundation

Dr. Roy Swank. Photo via Swank MS Foundation

This diet was first developed and tested in 1949 by Roy Swank, a distinguished academic neurologist at the University of Oregon. Swank’s research was rediscovered and refined, beginning in 1999, by George Jelinek, an eminent Australian doctor and professor whose new research facility at the University of Melbourne I visited last fall.

Why our family wasn’t advised to try this simple, harmless lifestyle change by four sky-highly-respected neurologists, and why it appears nowhere in the published recommendations of leading MS organizations, is the subject of what follows.

It comes down to this: The medical establishment now considers new treatments effective only if “proven” through double-blind randomized controlled trials (RCTs). In a blinded RCT, neither the researchers nor the subjects know who’s receiving an active treatment or a placebo, thereby removing many potential sources of bias. A researcher may not realize how his own preconceptions or hopes are influencing his observations; a patient’s symptoms might clear up if he believes his placebo is a real medication. But because people necessarily know what they are eating or whether they’re exercising, it’s impossible to conduct double-blind studies on dietary therapy or exercise. Even if a given illness could be wholly cured with diet or exercise, this would not be provable according to today’s standards of proof. The means of obtaining clinical proof is thus lopsidedly weighted toward testing drugs, because placebos can be tested against drugs in double-blind studies.

Clinical research evolved the way it did for sound reasons, but the new standards came, as innovations will, with complications of their own. It’s easy—and tempting, given the profits involved—to paint all this as the corporate conspiracy of Big Pharma, but the real problem is rooted in something else. We’re in the middle of a generalized epistemological failure brought on by a too-narrow understanding of “proof.” We’re in thrall to the tyranny of data. This deeper problem both weakens our clinical research institutions, and makes them vulnerable to charlatans and profiteers.

Bear in mind that the best available conventional medications for MS likewise offer only a chance, not a certainty, of slowing or halting the progression of the illness. There are other safe therapies, like a vegan plus fish diet, that offer, likewise, a chance. Anything that is safe, and that might help, you’d want to try. Potentially massive upside, zero downside, seems like an attractive proposition.

Nevertheless, the vast majority of those diagnosed with MS today all over the world are being told, in response to the question, “Are there lifestyle changes I should try? Are there dietary recommendations for me?”

“No, no there aren’t.”

But yes, you know: yes, there are.

 

II.

George, center, as a young man, with his mother and family.

George, center, as a young man, with his mother and family. Photo courtesy George Jelinek.

The first thing to know about George Jelinek is that his mother Eva died of multiple sclerosis; she was just 58 when she took her own life by an overdose in 1981, rather than continue to live with her increasingly grave disabilities. The second is that he is a prominent physician and academic, a co-author of the standard Australian textbooks on emergency medicine; that is to say, a highly qualified judge of medical research. And the third is that he himself was diagnosed with MS in April of 1999, but has been relapse-free for seventeen years.

After his diagnosis, which rocked this very driven, focused man to his foundations, Jelinek took six weeks off work in order to reassess his situation, his career and his future. In the course of reviewing the MS literature he came across a 1990 Lancet paper, “Effect of low saturated fat diet in early and late cases of multiple sclerosis.”[1] Its author, Roy Laver Swank, was then at the end of a long and illustrious career. A former Head of Neurology of the University of Oregon Medical School (now Oregon Health & Science University), Swank had published research on combat exhaustion in WWII; the effects of fat on blood viscosity; thiamine deficiency; and the prevention of microemboli following heart surgery—170 papers in all. (An oral history he gave in 1998 offers an absorbing account of his life and work.) The Lancet paper was the culmination of more than four decades of research on the relationship between saturated fat and the disease course of MS.

In 1948, the pioneering neurosurgeon Wilder Penfield offered Swank an assistant professorship at the Montreal Neurological Institute, which had just received a big grant earmarked for the study of MS by a local industrialist whose son was afflicted with the condition.

It had by then become evident that certain populations were far more prone to developing MS than others. Swank traveled to Europe, where the war had created long, well-documented periods of extreme lifestyle changes that might yield more clues. And in Norway, he learned that the incidence of MS in remote, mountainous dairy-farming regions, where there was a high consumption of beef and dairy products, was eight times higher than on the coast, where the predominant diet was based on fish.

Swank’s early findings were published in the May 8, 1952 issue of the New England Journal of Medicine (“Multiple Sclerosis in Rural Norway: Its Geographic and Occupational Incidence in Relation to Nutrition.”) The paper makes for exciting reading. Among many other investigations, his team measured or weighed every iota of food that came into the houses of more than a hundred Norwegian families in seven rural districts. Though his samples were small, Swank maintained that the populations he’d studied were stable, and that the patterns observed were consistent over a long period. He concluded by noting that the marked tendency of chylomicrons (lipid-transporting particles) to clump in the blood after a meal containing butterfat might lead to an explanation of how MS is triggered in susceptible individuals.[2]

When he got back to Montreal, Swank set up a study to test his theory that ended up lasting for more than 35 years. He put 150 Canadian MS patients on a mostly-vegan diet, with strict restrictions on saturated fat. He published a number of papers in the intervening decades, and in 1972 brought out a mass-market book: The Swank low-fat diet, a version of which is still in print.

From quite early on, the establishment tended to take a dim view of Swank’s work. In a 1961 review of the monograph, “Multiple Sclerosis and Dietary Fat” in the august British Medical Journal, neurologist Henry Miller scoffed openly:

[Swank’s] account of the factors which influence the severity of the disease rests entirely on clinical impression […]

Before neurologists will join their cardiological colleagues in attempting to bankrupt the dairy industry—and modestly correcting dietary habits of several thousand years’ standing—they will need more convincing evidence that the differential geographical incidence both of multiple sclerosis and atheroma is not due to Protestantism, central heating, spirit-drinking, woolen underwear, or some parallel concomitant of life in a cold climate […] The author is riding a hobby-horse rather than dispassionately seeking the truth […]

This seems a careless reading of Swank’s research, a key point of which was that Norwegians, whether farmers or fishermen, had quite a lot of lifestyle features in common (including Protestantism, spirit-drinking, and central heating, it seems safe to say), and yet there was this huge differential incidence of MS between them. Also, the notion that the interests of the dairy industry would be of the remotest concern to anyone seeking an effective treatment for multiple sclerosis is perfectly bizarre. I read this thing half a dozen times and just had to pack it in, finally. Put it this way: Roy Swank, who followed his own damn diet, died in 2008 at the age of 99, having outlived Henry Miller, who was four years younger than himself, by 32 years.

In any case, Swank’s Lancet paper reported that half the 144 members followed in the 34-year study had stayed on the diet, and the other half hadn’t, in varying degrees. The majority of those who had stuck with it were, comparatively, in fantastic shape. Of those entering the study with minimum disability, 95 percent were still ambulatory and active at the end. For the “good dieters,” relapses had plummeted from an average of around one per year to .05 per year. But 58 of the 72 “poor dieters” had died by the study’s end, 45 of them from MS-related causes.

The difference between the two groups was far too extreme to interpret any other way; the central nervous systems of those who’d avoided saturated fat had been preserved, and their lifespans greatly extended.

In a 1998 interview, the nutritionist and physician John McDougall asked Swank, who was then 89 and even less of a fool-sufferer than he had been as a younger man, why the National MS Society had not adopted his recommendations.

Most people in this country expect to be cured by a pill, and have a cure that is almost instantaneous. With the low-fat diet, the people actually have to work to get better, and have to cure themselves. And as far as the MS Society, John, they don’t mention it because they didn’t discover it. It wasn’t their research dollars that found this treatment. So they’re not going to tell anybody. I discovered it in my small office here, in the basement of the University of Oregon Medical School […]

I was more or less considered a quack. I wondered why doctors felt this way. I think one of the reasons was economics. It doesn’t cost much to eat well and take care of yourself with a highly vegetarian diet.

To return, then, to George Jelinek, who in 1999 was wrestling with the likelihood that he himself would soon be suffering, as his mother had, a humiliating deterioration into paralysis, catheters, bedpans, and total dependency—plus, perhaps, cognitive impairment, and a host of other miseries. It was the worst possible nightmare for the dynamic golden boy, surgeon, professor, administrator, husband and father, whom colleagues had nicknamed “Action Man.” His youngest child, Pia, was just seven.

George Jelinek in his offices at the University of Melbourne.

George Jelinek in his offices at the University of Melbourne.

Jelinek recognized at once that Swank must have been onto something. But much more had been learned in the nine years since the Lancet paper’s publication, about the diagnosis of MS, about demyelination of nerves, about nutrition and lipid metabolism. He began the diet himself, and set to work. Forty-nine days later he had written his first book on the subject: Taking Control of Multiple Sclerosis: Natural and Medical Therapies to Prevent Its Progression.

Fifteen years or so after that, in my own state of emergency, I bought a copy of Jelinek’s subsequent book, which came with an even more confident-sounding title: Overcoming Multiple Sclerosis: an Evidence-Based Guide to Recovery. This new book, like the first one, promised no cure. Basically it said: here are some things you can do that will likely lessen your chances of getting sick: a dietary regimen based on Swank, plus exercise and meditation, with evidence to support each recommendation. A rundown of the medications, with the exact pros and cons of each, so you can make up your own mind. Here, in short, was the book I’d asked for at the hospital. Jelinek neither condescends, nor leaves you in the dark; he gives you information, and work to do.

We went on the diet right away, read a million vegan cookbooks, and had fun making new things like almond milk and vegan enchiladas with chile cashew crema. It was an adjustment—I still miss cheese, a little bit—but we come from a cooking family, and soon found that there was always something delicious to make and eat. Being encouraged to take care of herself made a world of difference to Carmen, and to us. We weren’t waiting around for the next bad thing to happen; we were working together in a diligent and increasingly hopeful state of mind.

After a few weeks of reading, thinking, cooking, eating and talking with knowledgeable friends, Carmen was referred to Dr. F., a leading Los Angeles neurologist and researcher, in order to obtain a prescription for MS meds.

 

III.

Dr. F. turned up right on time with a couple of residents in tow, all in regulation white coats and radiating smooth, professional detachment. She is a tall, commanding woman with a booming voice.

“I’m an expert in MS, and that’s really what I do entirely,” she yelled. I liked her right away. Forthright, confident, she examined Carmen’s reflexes and balance, with surprising patience and delicacy for such an imposing person. We talked about the accuracy or otherwise of Carmen’s diagnosis, about conditions like B12 deficiency that can mimic MS, about steroid treatments. After a while the three doctors withdrew for a few minutes to examine Carmen’s MRIs, and came back saying: “Classic MS.” More talk about the disease process. Demyelination is like the peeling of the rubber casing on a wire, Dr. F. said. Questions for Carmen about her symptoms, and how she was feeling about starting meds. Carmen answered that she was planning to try Copaxone (glatiramer acetate), and added:

“The other big thing is we’ve been reading Dr. Swank and Dr. Jelinek’s books … My mom and I are both on the diet.”

“WHAT DIET. WHAT IS THAT?” (Dr. F is just so loud anyway, when she gets her dander up she makes a real racket.)

“It’s low saturated fat—Jelinek says no eggs, no chicken; Swank says you can have a couple of eggs…”

“Why are you on it?”

“Because…” Carmen hesitated for a second, and then plunged in. “The case that they make in the book, that there is research showing that it does help with lowering the chance of more flare-ups—”

“Yeah, that’s not really been proven. So. You can do it if you WANT, so. But it’s not really been proven, that it does have an effect on MS, okay?”

Tension rose in the room. Or maybe I just fancied that. It certainly rose inside of me.

“It’s hard to prove that,” she continued, “because it’s really complicated to control something that’s diet in a rigorous way, you don’t have a control and an intervention, it’s not like a pill and placebo. You can’t do it hardly, right? Diet’s hard: Maybe it’s just because it’s hard to do, but for whatever variety of reasons, it’s not really been shown that diet makes a big difference in MS.”

(Nope! I was not going to say anything nope nope nope.)

“I usually tell people not to change their lives… it’s enough to have MS, and then to start taking things away from people?!” She was openly exasperated now. “‘Oh, you should eat this,’ it’s not really necessary, why are we doing it?”

Oh no, yes, I am totally saying something. Something tentative, though—light, respectful.

“It’s not too stressful, this diet,” I said. “And if it might help? Why not.”

She turned to me.

“Well… I don’t think it’s gonna help that much, honestly, and I think if you want to enjoy something, you should. I’ll just give you an extreme example, okay, thirty years ago they told women not to have kids; that it would make their MS worse, okay? And they’re wrong. And they took that away from some people, and I think that’s terrible.”

“That’s … extreme,” I replied, almost despite myself. “It’s a little different from giving up ice cream.”

Everyone in the room excepting Dr. F. giggled a little nervously, there.

“I don’t believe in telling people something unless it’s true, and you know that it’s been proven. And it’s not really been proven. If you want to do it, you can, but … you know. It just hasn’t. So. Don’t limit yourself and do things that aren’t necessary, because you are going to have things that are limiting.”

Here she remembered herself; she had no way of knowing what Carmen’s disease course might or might not be.

“Anyway, not necessarily, but … that’s all. I just think it’s important to deal with what’s absolutely been proven, versus what somebody wrote in a book. You know. Maybe they’re trying to sell books. So. The science of the academic community and the papers on that have not been proven.”

Okay, I thought. And maybe you’re just trying to sell consultations at six hundred bucks a pop, or whatever. (I kid. I really believe all these people are doing their damnedest, Dr. F. included. But the intention of Jelinek, or of Swank before him, wasn’t “to sell books,” any more than she is just in it for the money.)

In any case, the question of absolute “proof” is of no interest to me. We are in no position to wait for absolute anything. We need help now. And incontrovertibly, there is evidence—not proof, but real evidence, published in a score of leading academic journals—that animal fat makes MS patients worse. It is very clearly something to avoid. In my view, which is the view of a highly motivated layperson whose livelihood is, coincidentally, based in doing careful research, there is not the remotest question that impaired lipid metabolism plays a significant role in the progression of MS. Nobody understands exactly how it works, just yet, but if I were a neurologist myself, I would certainly be telling my patients, listen, you!—just in case, now. Please stick to a vegan plus fish diet, given that the cost-benefit ratio is so incredibly lopsided in your favor. There’s no risk to you. The potential benefit is that you stay well.

But Dr. F, who is a scientist, and moreover one charged with looking after people with MS, is advising not only against dieting, but is literally telling someone (Carmen!) who has MS, yes, if you like butter, you should “enjoy” it, even though there is real live evidence that it might permanently harm you, but not proof, you know.

In this way, Dr. F. illustrates exactly what has gone wrong with so much of American medicine, and indeed with American society in general. I know that sounds ridiculous, like hyperbole, but I mean it quite literally. Dr. F. made no attempt to learn about or explain how, if saturated fat is not harmful, Swank, and now Jelinek, could have arrived at their conclusions, though she cannot prove that saturated fat isn’t harmful to someone with MS. The deficiency in Dr. F.’s reasoning is not scientific: it’s more like a rhetorical deficiency, of trading a degraded notion of “proof” for meaning, with potentially catastrophic results. Dr. F. may be a good scientist, but she is a terrible logician.

If she’s wrong about animal fats, the cost to Dr. F. (and to her patients) is sky high. But this really happened, exactly as I am telling you; a prominent specialist, a researcher at a massively endowed teaching hospital, advised my daughter to “enjoy” food that has a well-documented chance of making her really sick, disabling her, hastening her death by many years.

Yes, I am still mad! But even more than that, I am on fire for these things to change, and at once.

Carmen and Eric have been together for six years. They met in the laundry room their first week of Oberlin and have been pretty much inseparable since.

Carmen and Eric have been together for six years. They met in the laundry room their first week of Oberlin and have been pretty much inseparable since.

* * *

To my everlasting shock I found that it was impossible to get a single doctor to read and comment, on the record, on George Jelinek’s work. Four MS specialists in different parts of the country, including Dr. F., did not respond at all to multiple requests for comment. Several other neurologists refused specifically because they were not specialists in MS, and therefore considered themselves unqualified.

I note in passing that in the U.S., a doctor’s professional reputation is, necessarily, extremely fragile. Their various institutions (medical schools, hospitals, professional organizations, journals) are guild-like in the power and control they have over individual members. The profession is competitive. And so doctors really can’t afford to stick their necks out at all, which substantially impedes the flow of discourse.

After many weeks I was able to persuade one neurologist, Dr. X, to read Jelinek’s Overcoming Multiple Sclerosis. “Overall I would say that the theories underpinning OMS are plausible and the lifestyle modifications are reasonable,” he wrote in an email, a couple of weeks later. “While I cannot say the book provides definitive evidence of a therapeutic effect, I would likely advise a family member to follow many of the suggestions in this book.” Dr. X had some criticisms, mostly concerning the impossibility of randomizing the participants in the studies Jelinek and his team conducted in Melbourne.

Dr. X was concerned, too, about “blaming the patient for bad outcomes.”

There is always a risk that patients who progress will feel unnecessarily responsible for their outcome if we place lifestyle modifications at the center of a treatment plan. I believe this is a risk worth taking when we have solid evidence of a treatment effect (e.g. heart disease). There is less evidence here so this risk is a little more concerning.

Excitingly, he concluded with a request to show the book to a colleague, “one of the senior MS doctors” at his hospital. Are you kidding?! I replied. After so many weeks of trying to get any doctor to discuss this book with me, I couldn’t wait to hear more. But alas, that was the last I heard from Dr. X, or indeed from any of the doctors I begged to review and comment.

 

IV.

A growing number of scientists are convinced that the clinical research establishment is substantially broken. There are a number of systemic problems, but in my view they center in a wrongheaded notion of “proof,” like Dr. F.’s. We’ve known for a long time that science cannot produce absolute proof, and it was never meant to try; in the course of chasing absolute, unassailable proof from science, we have grown increasingly unable to identify right answers, or good ideas—the provisional truths that are all that science can ever supply.

Technically I guess you could trace the intellectual underpinnings of the question of “proof” back to Plato, but the philosopher Karl Popper put our current predicament very neatly in 1945. “In the empirical sciences, which alone can furnish us with information about the world we live in, proofs do not occur, if we mean by ‘proof’ an argument which establishes once and for ever the truth of a theory” [emphasis mine]. Pure mathematics and logic, Popper said—closed systems, which alone permit the possibility of absolute answers—”give us no information about the world, but only develop the means of describing it.” So, was he suggesting that science can’t help us solve problems in the real world? Not so fast.

“Although proof does not play any part in the empirical sciences, argument still does,” he said. “Indeed, its part is at least as important as that played by observation and experiment.” Science cannot provide proof, but it can construct actionable scientific theories.

In order to qualify as “scientific,” a given theory must permit of being proved false: this is Popper’s theory of falsification. Until you can prove that it is false, the best theory can, even must, be considered knowledge; provisional knowledge, to be sure, which is all we can ever get IRL, but knowledge nonetheless. For example, Einstein’s theory of relativity, which explains gravitation, was confirmed by Eddington’s eclipse observations in 1919. Had Eddington’s findings been otherwise, Einstein’s theory would have failed, and been discarded. This scientific method of knowledge production Popper contrasted with the theories of astrology, which, while they too depend on data and on minute observations of the physical world, can never conclusively be proved false.

Let’s propose for example the theory that the sun will rise tomorrow. It is safe to proceed on that assumption until the day that it does not, at which time you will be requiring a new theory (as well as a new planet, probably.) That the sun will rise tomorrow is not a fact; it is not “proven” and can never be proven; it is a theory based on the available information. But it is a very useful theory that can—that must, even—serve us as knowledge. We construct arguments based on the theory in order to advance other plans or ideas—such as, let’s figure out how to increase crop yields, or let’s go to the beach tomorrow.

In Montreal, Roy Swank made observations, took measurements, and then constructed an argument about multiple sclerosis which, while it cannot be proved, may yet be useful. But by narrowing the field of “proof” or reliable information to include only that which can be supplied by the results of randomized controlled trials, people like Dr. F. have lost the ability to evaluate the worth of a useful argument like Swank’s, let alone construct one. The growing intensity of commercial and regulatory pressures on scientific research has worsened the rhetorical climate still further. This is the source of our epistemological catastrophe in both the medical profession, and in the broader scientific community.

Fortunately, the realignment of scientific inquiry along more useful, pragmatic, Popperian lines has been coming for some time, a corrective reaction to the 20th century’s passion for an “objectivity” that turned out to be chimerical.

* * *

According to his obituary in the Financial Times last year, the legendary epidemiologist David Sackett began developing the modern practice of “evidence-based medicine” (EBM) in 1959, when he cared for a teen with type A hepatitis. Bed rest was then the standard protocol, but “this conventional wisdom formed the basis for daily confrontations between an increasingly restless and resentful patient and an increasingly adamant and doom-predicting clinical clerk,” he recalled in a 2010 paper. He is a fantastically entertaining and elegant writer. Anyway, Sackett went trawling through the literature for a proper explanation of the need for bed rest to share with this kid in order to persuade him to play ball, and stay in bed like he was supposed to.

Instead, he found compelling evidence to the contrary. The three-month convalescence prescribed for hepatitis A had been proving a “drain on military manpower” in the Korean War, so much so that in 1955 Tom Chalmers, a U.S. Army gastroenterologist, designed and performed a study randomly allocating soldiers afflicted with hepatitis to two groups: one assigned bed rest, and the other permitted to wander about at will; the two groups took an identical time to recover. Armed with this study, Sackett persuaded his supervisors at the hospital to permit his rowdy teen patient to romp all over the ward to his heart’s content, and he turned out just fine.

That is how David Sackett first became a champion of randomized controlled trials. But even though he is everywhere called “the father of evidence-based medicine,” Sackett thought there was more to practicing medicine than the application of relevant research. “Good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough,” he wrote in 1996.

Some questions about therapy do not require randomised trials (successful interventions for otherwise fatal conditions) or cannot wait for the trials to be conducted. And if no randomised trial has been carried out for our patient’s predicament, we must follow the trail to the next best external evidence and work from there.

Not long after Sackett wrote this, certain weaknesses in the RCT itself began to be identified in the literature.

* * *

In 1998, Harvard professor Ted Kaptchuk examined the metamorphosis of placebo after World War II. As it became evident that taking a placebo could result in significant patient responses (“placebo effects”), the notion of the “powerful placebo”—”an entity with occult-like powers that could mimic potent drugs”—gained widespread authority.

The evaluation of new treatments had once been made by doctors who arrived at their conclusions directly, patient by patient. But after the war, a contest developed between bedside and bench—between a desire for objectivity, precision, and consistency on the lab side, and the clinicians’ preference for working with real people. This tension played out through the development of RCTs, as claims of “objectivity” were tested and found wanting, again and again.

Now the placebo was assumed to produce a response of X, and the active agent, X plus the “real” therapeutic value. But that assumption, too, proved false. In a 2001 paper, “The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf?” Kaptchuk asked: Can an ‘unbiased’ method—a method by which every visible avenue of bias has been carefully closed off—produce bias?

The answer, he argued, is Yes. Even participating in a study alters a subject’s experiences from what they would otherwise have been, so that the very act of setting up controls might yield results specific to those new circumstances. The act of observation alters all we see, kind of like a Heisenberg’s (Clinical Research) Uncertainty Principle, throwing doubt on the idea that RCTs can produce incontrovertibly reliable “proof.”

It took a long while, but eventually researchers realized that any number of results differing from “no treatment at all” might be produced by placebo or an active agent in an RCT. The fear that you’re being given a placebo might worsen your condition. The “nocebo” effect was discovered, in which the anticipated negative effects of an active agent might harm a patient (for example, you might be given a placebo and warned of the “real” medication’s potential harmful side effects, and go on to experience them). But even with these deficiencies, Kaptchuk concluded, RCTs are a valuable tool.

It seems that the most “rigorous” evidence may produce deviations from the “truth.” The claim that the RCT is objective may fall short of a “hard” correspondence with reality. Still, the blind RCT may be objective in a “softer” or disciplinary sense: it is a standardized, explicit, replicable, and impersonal procedure that defines unambiguous and formal norms for medical researchers.[3]

In other words, the RCT is useful: it can be employed to develop a working theory, one that scientists can adopt, not as “truth,” but as Popper suggested, as the best theory we can construct, based on rigorously produced, replicable results.

The key point here is that the limitations of any investigative strategy must be taken into account, in medicine or anywhere else. A conclusion that nobody can draw, however—whatever Dr F. may like to think—is that a working theory can only be valuable if it can be tested within the narrow and somewhat flawed parameters of modern scientific publishing.

"Cartooning and teaching art to kids are Carmen's big passions. She made this drawing of us when she was living in Paris during her junior year of high school."

Cartooning and teaching art to kids are Carmen’s big passions. She made this drawing of us when she was living in Paris during her junior year of high school.

* * *

The ranks of scientists demanding reforms in clinical research are swelling, year by year. The best-known among them is the Greek epidemiologist John Ioannidis, who has been exposing the failings of the scientific establishment for 20 years or more. A colorful example of his team’s accomplishments came in 2001, when a rumor began floating around Greek hospitals that surgeons had been performing unnecessary appendectomies on Albanian immigrants just for practice. Ioannidis challenged a young doctor named Athina Tatsioni to prove it. Tatsioni set to work on a set of hospital records, and with help from Ioannidis and his group, published a paper showing that healthy appendixes had been removed from patients with Albanian names at a rate three times higher than those removed from patients with Greek names.

Ioannidis gained worldwide fame and a high position at Stanford mainly on the strength of his 2005 paper, “Why Most Published Research Findings Are False.” Here he argued that researchers are so pressured to secure funding, and to publish newsworthy findings quickly, that the true goal underlying most studies is the advancement of researchers’ career imperatives. He illustrated these claims with compelling statistical models. The frequency with which apparently irrefutable conclusions were later disproved, he surmised, threw serious doubt on the RCT process itself. He has been developing this line of inquiry ever since.

“At every step in the process, there is room to distort results, a way to make a stronger claim or to select what is going to be concluded,” Ioannidis told The Atlantic in 2010. “There is an intellectual conflict of interest that pressures researchers to find whatever it is that is most likely to get them funded.” In a subsequent PLoS paper he called for modifications in the “reward system” for science, so that the distribution of promotions, administrative power, grants and other academic valuables would be “better aligned with translatable and reproducible research.” There are many other notable clinicians and researchers calling for similar reforms, including Marcia Angell, Jerome Kassirer, and Peter Wilmshurst.

In an accompanying op-ed, “Minimizing mistakes and embracing uncertainty,” the editors of the PLoS who published Ioannidis’s 2005 paper threw their weight behind his conclusions, but with certain caveats. “Scientific truth is a moving target,” they wrote. “Even if one assumes no bias, wrongdoing, or ignorance on the part of any of the individuals involved—which is unrealistic, no doubt—chances are that some findings will turn out to be false. But is it inevitable, as John Ioannidis argues, […] that the majority of findings are actually false?” Probably, they agreed; Ioannidis’s argument was convincing. So: how to fix things?

Waiting to publish until absolute certainty is achieved would undermine the whole point of scientific research, the editors argued. “Progress depends on dissemination of results, and journal articles are the most effective tool we currently have to share them. The answer, therefore, cannot be that we wait until conclusions are proven beyond a doubt before we publish them. Publication of preliminary findings, negative studies, confirmations, and refutations is an essential part of the process of getting closer to the truth.”

We acknowledge that most studies published should be viewed as hypothesis-generating, rather than conclusive. And we publish high-quality negative and confirmatory studies.

We also accept some responsibility for educating consumers of research about the research process. Consumers also need to become comfortable with uncertainty, and understand the strengths and weaknesses intrinsic to every study conducted and published.

Very sound conclusions, no doubt. And yet some of us consumers are still not being told any such thing. It’s been 10 years and more since the publication of these comments, and yet when you visit a neurologist, right now, to ask about treatment options for your multiple sclerosis, she may very well insinuate that published research equals proof. She may advise you that any therapy that “hasn’t been proved” should be ignored altogether.

* * *

Ioannidis’s most recent paper, “Why Most Clinical Research is Not Useful,” was published in June 2016.[4] You wouldn’t think a medical journal in 2016 C.E. would need to say something so blindingly obvious, and yet here we are: “Medical interventions should and can result in huge human benefit. It makes no sense to perform clinical research without ensuring clinical utility.”

Encouragingly, he sees a role for patients and patient advocacy groups, and for public media, in this effort. “These groups can influence positively the utility of research […] Public media and related commentators of health news may also help by focusing on the need to obtain clinically useful research.”

Consider this, then, the earnest demand of a member of the public media, and also a patient advocate.

Expand the idea of “usefulness” to include such unbiased information as can be gathered not only from research, but from every responsible source—from families, from journalists, and from literature produced according to standards both archaic and modern. Let’s develop resourceful, inventive new methods of identifying trustworthy information—with disinterested intellectual rigor, and with the shared desire to build a better future. The medical establishment, patients and patient groups, research professionals, pharmaceutical companies, media! There is more in heaven and earth than is dreamt of in your randomized controlled trials. Let’s join together to learn all we can.

 

V.

Melbourne is a lovely city. Every day I was there I saw many unfamiliar birds, my favorite being a fat grey one, salmon-breasted, cockatoo-like, poking around on the lawn not far from where I was staying. Lovely plump little fellow. He is a Galah, I later learned. The fruit, the trees, the flowers, the dawn chorus, all wildly exotic, glorious. In the mornings I had a long walk to the university, near the commercial center downtown where you can buy goth clothes, or Balinese carvings, or have a fancy Chinese lunch in a mad old restaurant tucked away in a mews near the glittering river. It’s Victorian, Chinese, and Australian all at once.

But best of all were the featureless offices of the Neuroepidemiology Unit of the Melbourne School of Population and Global Health at the University of Melbourne, where Prof. George Jelinek and his associates work. They are plain offices with white miniblinds, white formica cabinets and industrial carpeting, but hanging around with these people who are moving things forward with so much energy, intelligence and grace makes you feel so alive.

The meetings I sat in on with Jelinek, members of his team, and various biostaticians and specialists brought home—and this is dumb of me, I know—how blithely I once accepted the charts and statistics in any given book or article. The numbers, the tables seemed so inevitable, so obvious, as if they’d organized themselves in the course of measuring; in reality they are not self-organizing in the least; the sleek organization is constructed from a giant pile of chaotic data. What do you include or leave out? What counts as an anomaly, as a tendency, as a “group,” as a “conclusion”? You have to choose all these things. Make them.

“You have to realize, these wonderful papers you see in the New England Journal of Medicine—actually, behind the scenes there’s a mishmash of data, all over the place sometimes; it’s messy,” Jelinek told me. “Often it looks great in the publication, but when you’ve done enough of it you realize that there’s a mess behind most research that’s really sort of carefully cleaned; people impute things, and make assumptions, so that you get some sort of clean analysis.”

Jelinek, who is in his early 60s now, tall and slim, with questioning, attentive deep-set blue eyes, has a long history as an academic researcher. In 1990, the year Swank published his culminating paper in the Lancet, Jelinek published one in the Medical Journal of Australia: “Ten Years of Snake Bites at Fremantle Hospital.” Snake and spider bites had been a special interest. For years he was the acknowledged leader in his field: President of the Australasian Society for Emergency Medicine and editor of the journal Emergency Medicine Australasia—just this very dynamic rock star emergency medicine guy. But in 1999 he was diagnosed with MS.

“And suddenly that wasn’t me anymore. I was eating vegan food at lunch and meditating in the office if I was feeling a bit stressed. And people just sort of… ‘What happened to George?'”

* * *

George Jelinek is a co-author of the standard Australian textbooks of emergency medicine.

George Jelinek is a co-author of the standard Australian textbooks of emergency medicine.

Jelinek showed me the massive standard Textbook of Adult Emergency Medicine, which he edited, along with four others, for many years. “I mean, there’s thirteen hundred specialists or something in Australia who believe every word of what’s in there, because of my reputation,” he said. His latest book, a new edition of Overcoming Multiple Sclerosis, lay next to the big textbook. “But there are people out there who do bee-sting therapy and stuff, but don’t believe [the multiple sclerosis book], because they think it’s ‘fringe.'” He laughed, gesturing toward one book, and then the other. “What’s in here is just as rigorous as what’s in here.”

With the new edition of Overcoming Multiple Sclerosis, Jelinek is looking to win over his medical colleagues as well as those who’ve been diagnosed with MS. “We’ve got thousands, tens of thousands the world over who are following the program, and they still often get quite negative responses from their treating doctors,” he said. “Unless you influence it from the other end as well, [the program will] always stay somewhere on the fringe, and it won’t be in the mainstream.”

Broad strokes, what did you change in the new edition?

[On average] it takes [those with MS] maybe ten, fifteen, twenty years to get to disability. But you get people where it takes weeks. So what I did here was to say, look: everyone should adopt lifestyle change and reduce their risk factors. That’s across the board. The drug approach, though, needs to be varied depending on where you are on this continuum. So if you’re someone like myself, who is maybe never going get to any disability, is it sensible to put them on drugs and have all the side effects? Well, clearly not. Predicting who’s going to beat it is a different matter…

But if you’re on the other end of that spectrum, and you’re going to be disabled within three months, why wouldn’t you be on the strongest possible drug, as well as doing all this? And that is the bit I was not advocating. Or… I sort of was, but very halfheartedly. But actually, it makes perfect sense. Of course they would be prepared to take the risk of serious side effects, if they’re going be in a wheelchair in three months.

If there’s some chance.

If there’s some chance. And there is. There are at least three drugs that would [potentially slow the rate of disability even for someone with very aggressive MS]. Of course, one of them—for instance, one in fifty people get leukemia from one of these drugs. One in ninety-nine people will get progressive multifocal leukoencephalopathy (PML), that terrible brain affliction, from one of these medications. Another one will get thyroid autoimmunity. But—[there’s a chance] they won’t be disabled in ten years. And that’s a risk that a lot of people would be prepared to take, and I wasn’t giving due acknowledgement to that.

Correspondingly, Jelinek has also grown more empathetic toward neurologists. Before the advent of the new MS drugs, doctors had nothing at all to offer their patients, who were facing this incurable, maybe disabling illness; “diagnose and adios,” they used to say.

To see how important that was to [the treating neurologists], to suddenly have some mainstream medicine where they could say, “I can offer you this, and we know you’re going to have fewer relapses… If you’re like the rest of the groups in all these studies, you’re going to have better health outcomes from this drug”—I think for me to ignore that [in favor of lifestyle modification] wasn’t giving these guys any credit.

* * *

Jelinek often talks of the difference between “healing” and “curing.” His central message is to encourage people to take a more active role in the care of their own bodies.

There has been this kind of imbalance in power embedded in the doctor/patient relationship for so long, you know; all through my career, almost like a parent/child relationship. [But] there’s a whole generation coming through now where it’s becoming a more even relationship.

What is the root of that, do you think?

It’s this patriarchal sort of thing, because the surgeon actually almost literally does have the power of life and death in his hands. It’s one of the few areas where we really cure, where there is sort of almost magic in it, isn’t there? Someone can come in and you know that without treatment they are going to die and then a few weeks later, they’re back to their normal life. Even the staff in theatre treat surgeons that way; you watch the nursing staff and the support staff and they almost genuflect when the surgeons come into the room. It’s so powerful, that position. Everything is geared to making it perfect for the surgeon. So they might have some classical music playing. No one else gets a say in what music is playing in theatre. If they want a particular instrument, then it has to be there for them.

But equally there is a tendency on the part of the person receiving the treatment to project all those enormous qualities and hopes and outcomes onto that person. So it cuts both ways.

Enabling it.

Yeah. It enables that attitude and behavior to flourish because we, as a community, we have this unrealistic expectation of what doctors can do. And often an equally unrealistic expectation or understanding of how much control we have over our health.

And that’s the thing I’m really keen to promote, is to make people more aware of how much of their health is actually in their own hands. That’s transformative, the experience you have as a doctor when you get sick. That’s where it comes from.

For the first time in your life… I mean I was forty-five years of age and for the first forty-five years, something goes wrong, you go to the doctor and it’s sorted out. And this sudden realization, actually—there’s something now that’s happened that they can’t fix, that the system has got no answer for at all. And where does that leave you? It leaves you in a really empty, dark place.

But to get to the other side of that and realize—actually, I have the capacity to change my future, it isn’t in anybody else’s hands. That’s a… most people don’t get to that point in their lives.

Of course if you stay in that other place, and you have that expectation the system will look after you and always has the answer, then we end up where everyone gets overweight, gets high cholesterol, doesn’t exercise, does all these unhealthy lifestyle things and then goes to the doctor expecting that a prescription will sort it, the magic will be invoked again and they’ll be able to lead another twenty years of healthy life.

Like Marcia Angell and other American reformers, Jelinek has also been very outspoken against Big Pharma. The drugs used to inhibit MS flare-ups are all “specialty pharmacy” drugs, which come with some of the highest price tags in the United States: the cost starts at around $50,000 per year, and can reach $160,000 per year or more. Jelinek told me that the MS business alone, at last count, was worth $16 billion per year to Big Pharma. (That figure is projected to rise to $24.8 billion by 2024.)

How did it get that expensive? Do we even know, really?

In countries like America, the land of opportunity and free enterprise and all that stuff—there’s a lot of regulation favoring industry. In places like the UK, where there’s a much more socialist milieu, [the costs for MS drugs are far lower]. Australia is somewhere in the middle. We’ve got a socialist system but with a lot of this kind of capitalist, entrepreneurial side to it.

So the cost of [these drugs is] driven largely by America, because of course the FDA has this incredible power not only within America, but worldwide. People see it as the kind of governing body. And if the FDA approves something, then everyone else just falls into line. Europe can say: Oh, our regulators are independent, but actually once it’s approved in the U.S., it’s just a matter of time. The dominoes just fall around the world.

It always ends up the fox is guarding the henhouse.

Dracula in charge of the blood bank. It’s a real problem. There’s very celebrated cases where things have come to light; court cases, where you actually see the inner workings.

Like Vioxx.

Things like the Vioxx case, where instead of pulling it from the market they put a black box warning on it and it ended up killing a hundred thousand Americans…You get one death from a Chinese herb and it’s on the front page. And a hundred thousand Americans died from Vioxx.

* * *

When people are touched by a danger like MS that is so poorly understood, the need for a working balance between faith and skepticism suddenly crystallizes. You must choose a path, and set your feet steadily upon that path. Jelinek quoted to me the American author Michael Specter. “Be skeptical, ask questions, demand proof. Demand evidence. Don’t take anything for granted. But here is the thing: When you get proof, you need to accept the proof.”

When people like Jelinek, or psychiatrist Dr. Keryn Taylor, or pathologist Dr. Ginny Billson, whom I met in Melbourne, or any of those with MS who’ve been on the OMS program for many years, have recovered and no longer experience symptoms, this flies in the face of all conventional wisdom. It’s not possible to promise that the Swank or OMS diet is a cure, but without question these lifestyle modifications are worth looking into for anyone facing an MS diagnosis.

“We don’t have a test,” Jelinek told me, “unlike other things in medicine, to tell you how fast you’re going to deteriorate… There is no paradigm to think that [recovery] is possible. The average doctor, who has only ever seen people deteriorate, would think: But that is impossible. Without knowing all of this background, the understandable response is to say that it’s not possible, you must never have had the disease.”

It’s being trapped in your paradigm and it’s like… Who was it who said it? The three stages of acceptance of a new idea?

The first stage is everyone just says you’re frankly mad. The second stage is this grudging acceptance that it’s possible, there just might be something in it. And then the third stage is, oh, it was their idea. And that’s I think what is starting to happen; people are starting to enter the second stage. Once or twice now I have heard people even start being in the third stage and saying: Well, we’ve had an interest in lifestyle for a long time in MS…

Aren’t you prescient. That’s great.

[laughing]

You guys are just geniuses. I am so happy you’re researching this.

Good for you.

Yeah.

Take the lead.

Well exactly it, just—whoohoo. There’s no competition. This is just… you come to the party, we are all happy. You know?

The truth, I find it so… I am in this stage where I’m very mad, you know?

Yeah, still angry. I lost that a while ago.

* * *

There’s a recent Australian documentary called The Connection in which Jelinek appears alongside Dean Ornish and others. In the film, he says, “If you looked at the kind of cards I was dealt, I think the average person would have said that that’s a recipe for a disastrous life. And yet I don’t remotely feel like I’m living the kind of prescription that was delivered. It’s real hope that you can end up perfectly well after ten or fifteen years for this illness. I mean, I have.”

This is a good time to add that Carmen, who’ll soon celebrate her 24th birthday, hasn’t had a relapse in over a year.

Maria and Carmen, blowing some Bitcoin stash in Las Vegas.

Maria and Carmen, blowing some Bitcoin stash in Las Vegas.

* * *

Maria Bustillos is a journalist and critic living in Los Angeles.

Editor: Mike Dang; Fact-checker: Matt Giles

* * *

[1] Effect of low saturated fat diet in early and late cases of multiple sclerosis. Swank, R.L. et al., The Lancet, Volume 336 , Issue 8706 , 37 – 39.

[2] This question is still being mooted. In 2011, for example, Angelique Corthals, a forensic anthropologist and assistant professor at John Jay College, made a splash in the Quarterly Review of Biology advancing a theory suggesting that MS caused by faulty lipid metabolism, it much more closely resembles coronary atherosclerosis, or hardening of the arteries, than it does autoimmune disease: “When lipid metabolism fails in the arteries, you get atherosclerosis. When it happens in the central nervous system, you get MS. But the underlying etiology is the same.”

[3] The double-blind, randomized, placebo-controlled trial: Gold standard or golden calf? Journal of Clinical Epidemiology 54 (2001) 541–549

[4] Ioannidis JPA (2016) Why Most Clinical Research Is Not Useful. PLoS Med 13(6): e1002049. doi:10.1371/journal.pmed.1002049