Parameters of Efficacy

Shedding Light on the Subject

Imagine it's late in the evening. Really late. You're tired, but you can't sleep. Especially knowing that there's one last piece of apple cobbler left in the refrigerator — and it is calling your name with a voice that will not be ignored. You can't fight this one, so you drag yourself out of bed and head for the kitchen, slapping the light switch as you walk by without giving it a second thought. And why should you? Electricity has always been there when you've needed it in the past. You know that, unless there's a bulb burnt out, the light will work. When you turn on your coffee maker, you expect there to be coffee. When you're settling in to watch your favorite TV shows, you know they'll be there.

Besides, with the kitchen lit up and your eyes on the prize, the only thing on your mind right now is that tasty treat a few centimeters from your fingertips. The fact that something you expect to work is working is the furthest thing from your mind.

With something like electricity, there are no levels of effectiveness. It either works or it doesn't. In fact, we don't really care how it works, as long as it does what we expect it to do.

Measuring What Can't Be Seen

When it comes to medications, things aren't quite so black and white. If you have a headache, you probably know which pain reliever works best for you and will take care of the pain the fastest. The fact that it works says that the drug is effective at what it is supposed to do. In medicine, this is called the “efficacy” of a drug.

With some medications, showing they are effective is relatively simple. The efficacy of a pain reliever, for example, is demonstrated when your headache goes away. The efficacy of an antacid is demonstrated when your stomach stops feeling upset.

But what about in multiple sclerosis? There are events that occur in the body without us ever knowing. In fact, often what we feel are symptoms are actually consequences of an event that may have already occurred. How do you judge the efficacy of a therapy intended to regulate the disease process or change its course? How do you know it's doing its job when you can't always see or feel it working? For that matter, how do you know if it's not working, or is losing its effectiveness?

The clinical trial provides a way of demonstrating efficacy in a controlled environment. We'll look at what is known about how different treatments work, and what role or level of importance this knowledge plays in showing a drug's effectiveness. Finally, we'll look at efficacy in the context of the disease. Let's begin by defining one of the most important elements of any clinical trial: the evidence.

What Is Proof?

If you have been diagnosed with multiple sclerosis, chances are you've probably spent some time talking to other people with MS about their experiences with different therapies. Actually, this is a really logical place to start. You probably value their opinions as to how well different treatments worked and what type of side effects they had. After all, who would know better than people who have experienced something first hand?

If you could put together a list of notes from people who had experience with different treatments, this might seem to be enough to help you learn more about your treatment options — and which you may want to discuss with your doctor.

So why do researchers go through the trouble of setting up trials that take years, and wait to get the results back before filing a new drug submission for Health Canada approval?

While personal experience with a drug certainly is relevant, medicine learned its lesson the hard way in the early 1900's, when fad therapies and radical treatments for MS caught on more out of enthusiasm for the “new thing” than any real evidence that they worked.

In order to interpret what could scientifically be considered "proof", researchers agreed it was necessary to first determine what desired outcomes were being studied.

It's kind of like calling your shot in a pool game. If you call the 4, but sink the 2 and 7 in the same shot, how do your friends know if you're the shark you say you are, or just riding a lucky streak? Was it chance or ability that gave you the shot you wanted — and two bonus shots in the process? Could you do it again?

"Calling the shot" by defining the desired outcomes before a clinical trial begins is a way researchers determine if the clinical trial meets its objective. We'll use this analogy again later when we talk about outcomes, so keep your cues chalked and ready.

Clinical Trials

Measurable Outcomes

In clinical trials, a primary goal is to establish what is called a “measurable outcome.” Outcomes are simply the result of a process or action. And by being able to measure outcomes in clinical trials, we know whether or not a drug is working as intended.

Outcomes are just what they sound like: results. In fact, we deal with outcomes every day. An outcome of a football game, for example, is that one team wins and another loses. If winning was the “primary outcome,” it is measured by the final score.

But there are also “secondary outcomes” that teams agree will be measured during the game. Passing yards, an outcome measured by total yards of completed passes by the quarterbacks. Time of possession, another secondary outcome measured by how much of the total 60 minutes of play was dominated by either team. Rushing yards, an outcome recognized at the end of the season, even if the winner never makes it to a Grey Cup during his whole career.

All of these outcomes are recorded during a game. They may or may not have an effect on the primary outcome, but they still have importance.

In clinical trials, there are usually primary and secondary outcomes set as goals before a study even begins. This lets the researchers know two important pieces of information: does a therapy work, and what effect does it have? In medicine, this “clinical evidence” is what determines if a therapy is approved by Health Canada to be used and what claims it can make.

Statistical Significance

In clinical trials, outcomes are considered significant when the likelihood, or "probability", of these same results being repeated is considered high. This is calculated using a statistical formula, then assigned a value (represented as P=xx.xx) that tells other health care professionals and researchers how strong the data is that supports the claim. This is called a P-value.

Right about now you may be smelling a math lesson coming up and running for cover. Don't worry, the formula that's used to determine a P-value isn't what's important here. In fact, if you asked your doctors, chances are most of them wouldn't know the formula off the top of their heads themselves.

What they do know — and all you really need to know — is a single number. To be considered "statistically significant" in clinical terms, the P-value should be no greater than P=0.05. Anything lower than that is considered "significant", which to your doctors means that there is a good chance these results could be repeated. Anything higher than that says there is less of a chance the same results could be seen.

For example, P=0.38 would not be considered a statistically significant difference in an outcome. A result with a value of P=0.001 would be considered highly significant. This by no means is meant to say that treatments with higher values don't work. This is just another way of measuring the level of effectiveness seen in a particular trial.

See? Now there's something you can throw around at the next get-together with friends or your family and walk away feeling like a genius. Only you need to know it's actually fairly simple once you know which numbers are strong and which aren't.

Let's look at some of the outcomes that are used to determine the efficacy of different treatments, and how they help measure the progress toward achieving the major goals of MS treatment.

How Efficacy Is Measured

Let's take a few minutes to look at the primary goals of therapy and some of the common outcomes measured to help determine efficacy.

Controlling Relapses

Relapses by their nature can present problems when it comes to measuring efficacy and related outcomes, simply because they often can end all by themselves. Still, many treatments have been proven through clinical trials to have a positive effect on relapse rates.

Outcome measurements used to help demonstrate the efficacy of treatments intended to effect relapses include:

• A reduction in frequency: this means the trial was designed to study, among other things, a treatment's ability to reduce the number of times relapses occur over a set amount of time, typically a year. This is referred to as the “relapse rate.” While the natural course of MS shows that relapse rates become less frequent over time, a number of treatments have been shown to reduce relapse rates even beyond the natural course of the disease.
• A reduction in duration: achieving this outcome would show that a treatment would decrease the number of days that a particular exacerbation (relapse) would otherwise have lasted.
• A reduction in severity: lessening the intensity of an exacerbation is a goal of treatments, and an outcome that is measurable in clinical trials.

Altering the Course of the Disease

While more than a half dozen different scales for measuring disease progression have been proposed over the past 50 years, the gold standard — despite its limitations — has remained the Kurtzke (1983) EDSS scale. EDSS, you'll remember from previous studies, stands for the Expanded Disability Status Scale.

Because the EDSS scale is heavily biased toward mobility, it does not do a good job of taking into consideration other significant forms of impairment, such as vision loss or a loss of cognitive function. Unfortunately, this means that a person's true overall disability and impairment status aside from mobility can be difficult to map and follow scientifically.

Outcome measures related to EDSS scores that are used to show that treatments have a positive effect on disability include:

• Delaying further steps in EDSS scores: A significant goal of treatment is to slow progression of the disease, delaying the potential onset of a progressive phase as long as possible.

In clinical trials, efficacy of a treatment can be shown by measuring the average time before a group of people receiving the active drug (the treatment group) advances in the EDSS scale compared to the group receiving a placebo (the control group). The treatment has demonstrated efficacy in slowing the progression of the disease when its average EDSS is significantly lower than in the control group.

• Reversing EDSS scores: Certain factors in MS still not fully understood may make it impossible for impairment to be reversed in particular instances. Still, a reversal of EDSS scores has shown to be a realistic outcome that can be demonstrated in clinical trials.

Other ways of monitoring changes in ability associated with multiple sclerosis include:

• Standard neurological examinations: During the course of an office visit with your physician, this exam takes into account symptoms described by the patients, as well as observations and an analysis. The exam just before a treatment begins is important to help have a kind of “before” set of records, so the efficacy of any treatments that follow have something they can be measured against. This is called a baseline, which is nothing more than a base measurement.

Say for example you wanted to lose weight. Your "baseline" would be the first measurement you took before you started your weight-loss plan. This measurement simply gives you a starting point you can use to track your progress, and lets you know how well you're doing toward reaching your goal along the way.

• MRI: While MRI's main job is to help confirm a diagnosis, it is becoming more commonly used as a way to help measure theefficacy of different treatments.

MRI can assess the number and volume of new and total lesions as well as other metrics such as black holes and brain volume changes.

Symptom Management

Today there are a number of symptoms involved with multiple sclerosis which can be very effectively managed by different treatments. Anything from basic pain relievers for aches and pains, to anti-depressants for those people for whom the disease has taken an emotional toll. These and everything in between are available to help make living with MS a little easier.

With most symptoms, it's pretty easy to know whether or not a treatment is working. You feel better if it is, the same or worse if it isn't. You don't need science to tell you that. Still, the products wouldn't even be available had they not been tested, and certain parameters of efficacy set for them.

The Importance of "Sustained Efficacy"

The ultimate goal in any disease is finding a cure, plain and simple. Still, until that goal is achieved, the next highest priority is to find treatments that will continue to work year after year without losing their effectiveness. In medical terms, the ability of a treatment to keep working over a long period of time is called "sustained efficacy."

When it comes to treatments — especially those intended to affect the course of the disease and keep relapses under control — sustained efficacy is one of the single most important measurements of success. After all, a treatment that works for a while and then starts to become less effective is eventually no longer doing its job.

There are some people who were involved in clinical trials 20 years ago who may still be taking a certain therapy and are still seeing that it works. But once they have left the trial, how do we know whether it's the drug alone that's working or a combination of therapies and practices all working together to control the disease? We don't, unless we continue to monitor their results — which is exactly what some researchers do.

In clinical trials, there are usually two phases: a blind phase, and an open-label phase. If a clinical trial is "blinde", that means that neither the investigators, doctors nor patients involved knows who is taking active medication or placebo. Within the defined start and stop dates of a clinical trial, sustained efficacy can be difficult to prove. Most of the time, the “blind” phase of clinical trials is set up to go anywhere from a matter of months to a couple of years, so that once a drug has passed the necessary tests it can be made available to the people who need it.

Sometimes a group of people from the trial will agree to let their progress continue to be monitored well after the original part of the trial has ended. This is called the open label phase of the trial. This can be a tremendously valuable phase since it gives researchers a way to see how long the drug's effectiveness will continue — or, as science calls it, how long efficacy is sustained.

During this open label period, usually only the people who have been taking the treatment throughout the study participate. After all, who wants to stay in a non-treatment group for 10 or 15 years, and deny themselves any treatment all that time? In fact, whether or not to have people with MS involved in even a short trial and be part of the group not getting treatment is a hotly debated ethical issue.

When Efficacy Fails

A treatment only works if patients can tolerate it. No treatment works if it's left in the bottle. That's why it is hard to judge the efficacy of a treatment without first taking into consideration how regularly patients take it.

Even when a drug continues to be effective over time, sometimes either the persistence or seriousness of certain side effects may be enough to make a person quit taking it. When that happens, the efficacy of the drug means very little.

With some medications, there may come a time where the efficacy begins to fail. This may be because of the way the body's immune system responds to certain types of medication. There are some treatments that are more prone toward efficacy failure than others for this reason.

Sometimes knowing this and making a few changes, like changing the way the drug is administered, can help prevent it from losing its effectiveness. For example, giving high doses of steroid therapy in short (or pulsed) intervals instead of regularly over a period of years may lengthen the time before the body builds a resistance to their effect.

With other treatments, however, science still hasn't figured out a way to keep the body from interfering with their effect. In studies of beta-interferons, for example, the therapeutic benefit may decline over time in those patients who develop neutralizing antibodies. Neutralizing antibodies are proteins produced by the body that react to and destroy infection. Neutralizing antibodies may not appear for the first several months or even a few years after starting therapy. If they develop, the benefit of continuing treatment may decrease, to a point where it is like injecting an inactive placebo.

Fortunately, we live in a time where we don't have to make the kind of compromises people did even 20 years ago. If a treatment begins to fail, or side effects are simply more than a person is willing to take, today there are choices.