DrRich’s last post addressed a recent issue of the Archives of Internal Medicine which, strikingly, was largely dedicated to trashing the JUPITER study.
The JUPITER study was a landmark clinical trial in which giving the statin drug Crestor to apparently healthy individuals who were at increased risk of cardiovascular disease (and most particularly, had high CRP levels) resulted in a significant improvement in outcomes. In particular, within two years, individuals taking the statin had a 20% reduction in overall mortality, a 54% reduction in heart attacks, a 48% reduction in stroke, and a 40% reduction in venous thrombosis and pulmonary embolism. All these findings were highly statistically significant.
DrRich attempted to show that the criticisms of JUPITER recently offered by the Archives were sufficiently spurious to raise the question of what the authors and the editors were really trying to accomplish, and for him to suggest that perhaps they were auditioning for appointments to the government’s expert medical panels, which will soon begin determining who gets what, when and how. Indeed, DrRich will actually be quite surprised if none of these individuals end up with such an appointment. They have clearly demonstrated they have the right stuff.
Still, as DrRich also pointed out, the JUPITER study, while a reasonably straightforward clinical trial whose results seem impressive, was anything but air-tight. No clinical trial is air-tight, however, and if medicine were still practiced the way it should be, the JUPITER trial could be smoothly incorporated – with all its limitations – into clinical practice without a hitch.
But, since medicine is now practiced by guidelines, JUPITER poses a major problem. In fact, it has led to major and contentious debates between those who insist its results must be incorporated into formal clinical guidelines, and who insist they should not. On one hand, many point out that JUPITER is an important clinical trial which has demonstrated a vital clinical benefit (prevention of heart attack, stroke and death) with a high degree of statistical significance, which meets the high standards demanded by evidence-based medicine, and which therefore obviously demands a change in the clinical guidelines. But on the other hand, many others insist that the JUPITER trial simply does not demonstrate enough of a benefit with Crestor to justify changing the guidelines.
DrRich’s position – that the results of the JUPITER trial are striking and important but incomplete, and ought to change the conversation between, but not dictate the actions of, doctors and patients – simply does not obtain in the modern era.
So, unable to side with either party, DrRich observes with great interest the debate between those who want to change the guidelines, and those who believe that changing the guidelines would be the greatest of travesties.
Those who want to change the guidelines have, in their favor, the virtue of consistency. For, if one insists that every action by physicians must be supported by evidence-based medicine, then one is pretty much obligated to fully embrace legitimate clinical trials like this one that give clear-cut and statistically significant results. Unfortunately, the evidence-based strict-constructionists have painted themselves into a corner when it comes to JUPITER. They will not be able to say, for instance, “Statins are pretty much alike, so we’ll make the guidelines say ‘statins’ instead of ‘Crestor.’” For JUPITER did not study “statins,” it studied only Crestor, the most expensive statin on the planet. Expanding the results to all statins (despite a large body of experience that suggests this would be just fine) does violence to the whole concept of evidence-based medicine. It’s just not possible. The strict constructionists have therefore boxed themselves in to advocating a new, multi-billion dollar annual expenditure.
It is even more amusing to observe those who do not want to change the guidelines.
These people fall into two general camps. First, and easier to dismiss, are those who believe that drug companies are the embodiment of evil, and that any clinical trial sponsored by a drug company must be dismissed out of hand, particularly if the drugs which are being promoted are statins. (This, in fact, is the level of argument on which the main article in the recent issue of Archives relies.)
DrRich simply notes, once again, that the advancement of clinically useful medical science – in America and in the world – is almost entirely dependent on drug companies and other corporate dens of iniquity. That companies must pay for our medical research is the system we’ve invented. Furthermore, our total capitulation to the dictates of evidence-based medicine means that companies must fund large, expensive clinical trials like JUPITER before they are allowed to sell a new product, or to create a new indication for an old product. This evidence-based paradigm is inherently a double-edged sword. Sure, it creates a huge barrier to the development and adoption of expensive new therapies (which is the covert rationing dividend of evidence-based medicine), but it also creates opportunities, for companies who manage to successfully complete such trials, to create iron-clad indications for their products. For, once a product has been “proven” in a randomized clinical trial, there is no easy way to legitimately keep that product out of the guidelines and off the shelves. The makers of Crestor have simply figured out the rules. One can whip up anti-corporate emotions by criticizing the sponsor for playing the game well, but the fact that the sponsor stands to gain does not negate in any way the results of a well-designed study.
That the anti-pharmaceutical and anti-statin crowds vociferously object to the results of the JUPITER trial is, of course, entirely expected and cheerfully acknowledged. DrRich will merely observe that their position is one of default. It is not dependent on the scientific merit of JUPITER (or any company-sponsored study), and thus it adds no useful information to the debate. We can only note their objections and move on.
The second group of people who object to changing the guidelines are less dogmatic and more open to reason, and indeed (and very interestingly so) claim to be proponents of evidence-based medicine, and thus claim to be willing to follow the data to where it will lead. It seems pretty clear (to DrRich, anyway), that the chief concern of these individuals, as it relates to JUPITER, is cost. That is, this group feels strongly that the implications of the JUPITER trial are simply too costly to follow to their logical conclusion. This, indeed, is a very reasonable position to take.
Unfortunately, the only legitimate way to turn aside the results of a costly but statistically definitive, evidence-based study is by rationing healthcare. (To ration, remember, is to withhold at least some useful medical services from at least some people who would be likely to benefit from those services.) But we can’t do that, because, well, it would be rationing. Because members of this second group are unable to invoke the “r” word, they are therefore forced to find other “reasons” for keeping the guidelines unchanged. This unfortunate situation leaves them little choice but to discover ways in which to impugn the legitimacy of the JUPITER trial.
In short, they find themselves forced to engage in statistical legerdemain in order to diminish the significance of the JUPITER trial. There are several useful statistical arguments they can employ.
From what DrRich has seen, many of the arguments that have been ginned up to this end have not come directly from the JUPITER trial itself, but instead from an editorial accompanying this study, written by Dr. Mark A. Hlatky.
Most of Dr. Hlatky’s editorial is measured and reasonable. But he threw in a key summary sentence that has been greedily grasped by the anti-alter-guidelinetarians, to wit: “The proportion of participants with hard cardiac events in JUPITER was reduced from 1.8% (157 of 8901 subjects) in the placebo group to 0.9% (83 of the 8901 subjects) in the rosuvastatin [Crestor] group; thus, 120 participants were treated for 1.9 years to prevent one event.”
This statement, at least taken at its face value as a stand-alone analysis, is statistically naive and wrong. DrRich realizes that one or two of his readers might not enjoy statistical arguments, so if you do not wish to wade through the reasons why, simply skip the next two indented paragraphs.
In a long-term clinical study in which the endpoints are events that can occur at any time (such as heart attack, stroke or death), then the probability that an enrolled patient will reach an endpoint in the trial increases the longer he/she has been enrolled in the trial. But in virtually all clinical trials, the length of time different people are enrolled varies greatly. This is because it often takes years to enroll people in clinical trials, so that when the trial ends, some will have been in the trial for many years, others for only a little while. This means that the risk exposure of each research subject is different, and is proportional to the total time they were enrolled. Not uncommonly, the enrollment process is not smooth – there are periods of more rapid enrollment, and periods of slower enrollment – so if all you do is average the enrollment time (as was done by Hlatky – 1.9 years) you are likely to get skewed results. So it is simply not statistically legitimate to do so.
There is a legitimate way of analyzing such longitudinal outcome statistics, and it’s called the Kaplan-Meier method. And indeed, the authors of the JUPITER trial presented in their paper a complete Kaplan-Meier analysis of their data (see Figure 1 of their paper), and the results look quite a bit different from Hlatky’s summary statement. The Kaplan-Meier analysis reveals that the risk of heart attack, stroke, and death all increase steadily through at least 4 years (5 years was the longest time anyone was enrolled in this study), so that at 4 years, the risk of reaching one of the “cardiovascular event” endpoints was about 8% (not 1.8%). Further, the Kaplan-Meier analysis shows that the protection imparted by Crestor persists through at least 4 years, and that indeed the magnitude of protection (i.e., the difference in outcomes between the treated group and the placebo group) increases throughout that entire duration. So, at 4 years, the placebo group had roughly an 8% event rate, compared to roughly a 3% event rate for the Crestor group – an absolute difference of about 5% (not 0.9%). This is a far greater benefit than is suggested by Hlatky’s shorthand summary.
Suffice to say, then, that Hlatky’s summary statement apparently ignores the appropriately analyzed data which is clearly presented in the JUPITER paper itself, and which documents that the clinical benefit of Crestor was substantially more impressive than his widely-quoted summary statement suggests.
But as illegitimate as this summary statement may be, let us accept it at face value for a moment just for the sake of discussion, since that’s the data the anti-alter-guidelinetarians have latched on to.
Taking these numbers, the “antis” make the following argument: While the relative reduction in “hard cardiac events” is 50% (1.8 to 0.9), the absolute reduction is only 0.9%, which, anyone would agree, is a pretty small number. So, they conclude, the actual benefit imparted by Crestor is actually quite small.
That’s a very interesting argument. Let’s look at it in a couple of ways.
So we’ve got a population of patients whose risk of heart attack, stroke, bypass surgery/stenting, or death is about 2% at about 2 years, and by giving them a pill we can reduce that risk to about 1%, and we’re arguing that the absolute drop of 1% is not very much to crow about. Well, OK. But what if we found a pill that reduced their risk to zero at 2 years? That is, it completely wiped out the risk of cardiovascular catastrophes altogether. Would that be a good thing? Or would we say, “It’s just a 2% drop, really not much greater than the 1% drop we had with Crestor, so it’s no big deal?” DrRich thinks not. DrRich supposes we would think that totally eliminating all cardiovascular risk would be a very big deal.
When you’re starting at a 2% risk, then any drop in risk is going to be an “absolutely” small number. And if we’re not going to pursue improvements in outcome of such a small magnitude, then why the heck are we worrying about preventative medicine in the first place? Once you get past the big things (drain the swamps, don’t drink the water downhill from the outhouse, etc.) then all preventative medicine tends to consist of small, incremental improvements in outcome. Popular pronouncements to the contrary notwithstanding, preventative medicine is largely the art of spending a lot of money for this kind of incremental improvement. If we decide we shouldn’t do this anymore, then DrRich would find it unfortunate but understandable. But it hardly seems reasonable to arbitrarily focus on this one, particular improvement in preventative cardiology, and (within a healthcare system that insists it is not rationing care) pronounce that this is the one we’re not paying for.
Another way of looking at this “the benefit is too small” argument is by considering that 7.4 million Americans fit the entrance criteria for JUPITER. By giving all these people a statin, we would be preventing about 66,600 major cardiovascular events over a 2 year period. If you’re going to say that 1% is a small number, DrRich will counter that 66,600 is a big number. So do statins offer a substantial benefit or not? It depends on whether you choose to focus arbitrarily on the 1% or the 66,600.
(DrRich understands that many of his readers are not focusing at this moment on the 66,600 cardiovascular catastrophes that could be prevented, but on the 7.4 million people who will be taking a drug that costs $120 per month. But we’re not talking about cost yet, we’re only talking about whether the drug does some good. If we decide it does, then we’ll need to link that “good” to a procedure that measures whether the “good” is worth the money we would need to spend to achieve it. The “antis” try to avoid talking about cost – since that would admit they’re rationing – by insisting that there’s just not enough “good” to bother with. DrRich is simply pointing out that such an argument – that preventing 66,600 very bad outcomes is not enough to bother with – is on its face absurd.)
Another argument invoked by the anti-alter-guidelinetarians is based on the “number needed to treat” (NNT) analysis. Again they rely on Hlatky’s unfortunate summary of the data: “120 participants were treated for 1.9 years to prevent one event.” This number – which the “antis” insist is just too high – is misleading for the reasons already discussed. The real NNT, based on more legitimate statistical analysis, is plainly laid out in the JUPITER paper itself. It turns out that the longer patients in this trial were treated with Crestor, the lower the NNT became. So: At 2 years, the NNT was 95; at 4 years, it was 31; and at 5 years, it was projected to be only 25. Whether you think it is reasonable to treat 25 people with a pill for 5 years to prevent one of them from having a heart attack, stroke, or death is, DrRich supposes, a matter of opinion. But based on NNT analyses for many widely-accepted therapies in medicine today, it looks pretty good.
All these arguments, of course, are merely distractions. The fact is that JUPITER showed a pretty striking reduction in nasty cardiovascular events over s pretty brief period of time, and the only real reason there’s any controversy at all is because of the cost of Crestor.
That cost is what makes us want to withhold Crestor, even though it is imparting at least some (and, DrRich, argues, quite a bit of) clinical benefit. In other words, the high cost makes us want to ration Crestor. The fact that we can only ration covertly, instead of openly, is what makes us want to bastardize the science and do a Kabuki dance with the statistics.
If we were rationing healthcare openly, then we could do an objective, full-bore cost-benefit analysis on the use of Crestor in JUPITER patients, using legitimate and not ginned-up statistical analysis, and taking into account not only the cost of the drug, but also the cost that would be incurred by failing to stop preventable heart attacks, strokes, etc., and then determining where the overall cost-benefit result fell within our coverage criteria. If it met the criteria we would cover it, if not, not. This decision would not be arbitrary. It would be a fully transparent process, so that if the sponsor did not like the results, they would try diligently to find a way to reduce the cost of Crestor (DrRich thinks they would succeed) to a value that would be compatible with their staying in business. (And for the first time, the price of medical products would be determined by a Laffer-like curve, where a price that was too high – like taxes that are too high – would reduce revenue, instead of increase revenue. Companies, being fairly rational, would ratchet their prices down to the optimal price point.)
But since we insist on doing our rationing covertly, DrRich is sorry to say that we’re destined to keep making spurious arguments, and using dumbed-down statistical analysis to back them up. The JUPITER trial, while it is imperfect and while it does not answer every question, really is pretty straightforward. That we get so wrapped around the axle trying to fold such clinical trials into our covert rationing paradigm is simply another demonstration of the fact that covert rationing corrupts everything it touches.
DrRich explains it all in, Fixing American Healthcare – Wonkonians, Gekkonians and the Grand Unification Theory of Healthcare.