November 12, 2008

Letter Opposing Potential Approval of CorCap Cardiac Support Device

Daniel Schultz, M.D.
Director, Center for Devices and Radiologic Health
Food and Drug Administration
9200 Corporate Blvd.
Rockville, MD 20850

Dear Dr. Schultz:

Public Citizen’s Health Research Group writes to you today to express our deep concern about the Food and Drug Administration’s (FDA’s) ongoing review and possible approval of the CorCap (Acorn Cardiovascular) cardiac support device.  The device is a mesh support sock sewn around the heart during open-heart surgery for dilated cardiomyopathy, a form of heart failure in which the heart becomes enlarged.  Following the completion of CorCap’s pivotal randomized controlled trial, the device received a 9-4 recommendation against approval from the FDA’s Circulatory System Devices Panel,[1] three non-approvable letters from the FDA itself,[2],[3],[4] and a 3-1 recommendation against approval from the Medical Devices Dispute Resolution Panel.[5] These repeated rejections reflect a consensus that the device has failed to demonstrate safety and efficacy. 

Despite this history, the FDA has counseled Acorn how to conduct a new study and to submit another pre-market application (PMA) for CorCap, potentially without further review by an Advisory Committee.[6] Unfortunately, the design of this new study, recently posted on the federal clinical trials registry,[7] is so poor that it is unlikely to provide reliable data that would contradict the negative findings of the data so far submitted to the FDA. 

Background

Heart Failure

CorCap is designed for use in adult patients with dilated cardiomyopathy, a condition in which the failure of some muscle fibers leads other fibers to compensate by lengthening.  Unfortunately, this process of heart “remodeling” leads to decreased efficiency; the volume of blood pumped per unit of expended energy is diminished.  Some researchers hypothesize that remodeling itself could also be an independent contributor to progressive heart failure.[8] Theoretically, the CorCap device prevents or reverses remodeling by providing external support for the heart.

Previous attempts at reverse-remodeling, or reducing the size of the heart to improve efficiency, have been primarily surgical.  Unfortunately, surgical resection of portions of the left ventricle has not demonstrated comparable benefits to medical therapy and has actually produced adverse effects on measures of heart function such as the frequency of recurrent heart failure.[9],[10] Thus far, such physical reverse-remodeling has not been shown to be a safe and efficacious technique for alleviating heart failure.[11]

Pivotal Trial

In December 2004, Acorn submitted one randomized controlled study of CorCap as the pivotal trial in its PMA.  This trial had been the focus of considerable controversy within the FDA for nearly four years, even as the trial proceeded.  Acorn resisted the FDA’s suggestions for increased sample size, extended trial length, and coherent endpoint design.[12] FDA reviewers noted Acorn’s “suboptimal trial conduct,”[13]including enrolling patients before the final protocol was established and multiple other protocol deviations. These deficiencies are discussed at greater length below. 

As a result, the pivotal trial was difficult to interpret due to an array of problems including unblinding and large amounts of missing data. The FDA decided to seek advice from outside experts on CorCap by convening an advisory committee meeting. 

Advisory Committee

On June 22, 2005, the Circulatory System Devices Advisory Panel voted against approving the device 9-4. Panel members commented that the trial was so poorly conducted that no evidence of efficacy could be found. For example, panelist Dr. George W. Vetrovec concluded, “It seems to me that to present this degree of missing data in such equivocal results is just difficult to justify approval.”[1] The FDA followed with its first non-approvable letter dated August 12, 2005, emphasizing that it did not see a “reasonable assurance” of safety or efficacy in the original study population, the legal standard for device approval.

“Focused cohort”

Having been turned down by the advisory committee, Acorn decided to look for a subgroup of patients who had shown improvement on the primary endpoint by retrospectively analyzing the existing data.   After examining 152 combinations of baseline characteristics and clinical outcomes, Acorn selected the subgroup that had achieved the greatest difference between treatment and control on the primary endpoint: patients with only moderately enlarged hearts (left ventricular end-diastolic volume indexed to body size, or LVEDDi, of 30-40 mm/m2.[14] The original trial had allowed enrollment of patients with LVEDDis of greater than 30 mm/m2.)   Acorn submitted a revised PMA with clinical indications restricted to this “focused cohort.”   The FDA again declined to approve the device, issuing a second non-approvable letter on February 2, 2006, that cited the lack of new data and risk of false-positive findings from mining the data for positive responses.  The letter concluded, “we believe that the least burdensome approach is a prospective study that clinically validates the risk-benefit profile of your device.”[3] Acorn responded at the time that “the small confirmatory trial described by FDA would not produce scientifically valid or robust data.”[15] Ironically, this is the trial Acorn is currently conducting.

Dispute Resolution Panel

To avoid doing this small trial, Acorn decided to utilize a rarely invoked regulatory provision: it appealed to a Medical Devices Dispute Resolution Panel (MDDRP), an external panel intended to resolve “scientific disputes” that arise between device sponsors and FDA.[16] Only two other devices have ever come before a MDDRP.

One product, Intergel, was determined by the MDDRP to be approvable in a 4-0 vote.  Less than two years after subsequent FDA approval, the device was removed from the market due to dozens of reports of post-operative pain requiring repeat surgery, foreign body reactions and tissue adherence, including three deaths.[17] The MDDRP concurred with the FDA that the other device, the Revelation Tx Microcatheter, was not-approvable in a 5-0 vote. It was never approved.

In December 2006, the MDDRP voted 3-1 to not approve CorCap.   As panelist Dr. Warren K. Laskey concluded, “I don’t think there’s any way to salvage the current—what we’re looking at on the table right now…short of an additional trial.”[18] Before adjourning, the panel was asked what could make the PMA approvable in the future. Suggestions included an additional, larger trial that showed lower mortality and “clinically meaningful benefit.”[19]

New “Confirmatory” Trial

Acorn was left with no choice but to conduct the additional trial or abandon the device.[4]

In May 2007, the company announced that the FDA “would consider approval” of CorCap based upon a 50-person trial.[20] During a meeting with the company, FDA officials suggested that unspecified “scaled back statistical criteria” could be sufficient for approval.[21]   Further, the company stated, “if Acorn meets the success criteria for the confirmatory trial, it is expected that FDA may be able to approve the device without engaging a third advisory panel.”[6] This would mean that the device could be approved without the opportunity for the public to review the new data.

The Pivotal Trial

Any PMA for CorCap should rely heavily on the original pivotal trial data, as the bulk of evidence was generated during this trial and it is the best-designed study of the product. The many deficiencies in its execution thus deserve particular attention.

Methods

The pivotal trial enrolled 300 patients with dilated cardiomyopathy and class II-IV heart failure (IV being the worst) as measured by the New York Heart Association scale (NYHA). All patients received optimal drug therapy during the trial. The patients were stratified into two arms based on their need for mitral valve repair (MVR) or not, with nearly two-thirds of all patients assessed as needing the valve repair.  Within these arms, patients were randomly assigned to treatment with the CorCap device or not.  The patients were followed for 12 months.  The primary endpoint was a composite of 1) change in NYHA class, 2) the need for a major cardiac procedure (MCP) due to worsening heart failure, and 3) mortality.   The composite outcome was “improved,” “same” or “worsened.”  The secondary endpoints assessed 16 measures of patient function and heart structure.

Efficacy

• Missing baseline NYHA data leads to post-hoc imputation of data

Assessment of NYHA stage is a subjective evaluation of the patient’s capacity for a certain level of physical activity based on unvalidated questions such as “how often do you walk up and down stairs?”[22] Despite this, Acorn proposed that unblinded clinicians determine NYHA class. The FDA insisted that NYHA should be evaluated using blinded surveyors to avoid biased results.  

Unfortunately, Acorn had already enrolled nearly 60% (172) of its patients before the protocol that included blinded NYHA determination was established. As a result, none of these patients had a blinded baseline measurement for NYHA, heavily compromising the validity of one of the components of the primary endpoint.[13] To mitigate the absent data in this component of the primary endpoint, the FDA suggested, but did not require, that a statistical technique known as multiple imputation be used to estimate the missing data. Acorn initially refused, arguing that imputation was “highly unusual given the volume of baseline data that would be imputed,” and that the results “would not be looked upon as a clinically valid test by the medical or scientific community.”[23] But two months after data collection was complete and the results thus available to the company, Acorn decided that multiple imputation was in fact the best way to recover the missing data and proceeded with that analysis.[24]

Imputation of missing data for patients without a blinded baseline NYHA class turned out to be just enough to allow Acorn to declare that its overall composite endpoint was significant, bumping the p-value from p=0.12 when the baseline assessment was based on unblinded NYHA class assessments to a significant p=0.024 (OR=1.73) with imputation.   Unless otherwise noted, the pivotal trial results described below include imputed data for missing blinded baseline NYHA class assessments.

• NYHA class results are not clinically significant

During the trial-planning process, the FDA requested that Acorn propose clinically meaningful success criteria for several endpoints in its study.  Acorn initially complied, but the criteria were removed during later revisions of the protocol.[13] In the end, none of the proposed success criteria were met.[13] For NYHA class, Acorn had stated that only an average change greater than 0.5 would be clinically significant, but even with imputation, the absolute difference between the treated and control groups was only 0.07 on the 4-point NYHA scale.[25]

• MCP analysis subject to bias due to unblinding

The clinicians who made MCP referrals for study patients were aware whether or not the patients had received CorCap.  This was a problem of some concern to the FDA because clinicians may have been more reluctant to refer CorCap patients for MCPs, biasing the results in favor of CorCap.  It was known before the pivotal trial began that CorCap placement could result in severe adhesions (internal scar tissue) that could complicate further surgery.[23] Acorn even revised its instructions for physicians to include the difficulty of reoperation in April 2003, shortly before enrolling the last patient in June 2003.[26]

Of the three primary endpoint components, this leaves only death as an objectively determined endpoint. There was no difference between treatment and control groups on this outcome.

• Incoherent  composite endpoint

Although Acorn declared the primary endpoint to be statistically significant (p=0.024), the FDA disagreed, calling the primary endpoint result “not interpretable.”[27] One factor complicating interpretation was that, even if the components had been determined objectively, the composite endpoint combined outcomes of varying severity and clinical significance. For example, a patient improved if NYHA class decreased and did not receive an MCP or die. A patient “worsened” if the patient underwent an MCP or died.  In this scheme, both death and a single-point increase in clinician-assessed NYHA status were considered “worsened.”  Moreover, a two-point decline in NYHA class was statistically equal to a one-point decline.  Such conflation of unequal outcomes is a defining feature of poor endpoint design.[28]

• Primary endpoint not statistically significant in the MVR stratum

The patients who received CorCap in the MVR stratum showed no improvement over control in the primary endpoint (OR=1.51; p=0.17).[29] Although this study was not designed to power primary endpoint significance in the MVR and no-MVR arms separately, the smaller, no-MVR arm did show significance while the twice-as-large MVR arm did not. Ironically, Acorn’s follow-up study focuses on the MVR patients in whom no statistical significance was demonstrated, even after imputation.

• MCP component in the no-MVR stratum drives primary endpoint results

Of the six strata (two surgical strata * three endpoint components), the only primary endpoint component that showed any sign of benefit from CorCap was MCPs in the no-MVR stratum (OR not given, p=0.003).   MCPs in the MVR stratum were not significant (p=0.12) and showed a difference in the rate of MCPs between CorCap and control of less than 5%.[27]

• Insufficient multiple hypothesis adjustment for the secondary endpoints

Because testing several hypotheses increases the chance that one of them will be statistically significant merely due to chance, adjustment for multiple hypothesis testing should ideally be specified a priori. Acorn only specified multiple hypothesis adjustment for four of the sixteen secondary endpoints designated as “major secondary endpoints.” These included two measures of heart structure and two measures of improved patient function: left ventricular end diastolic volume (LVEDV), left ventricular ejection fraction (LVEF), unblinded NYHA, and Minnesota Living with Heart Failure (MLHF) survey, respectively.[15] Following the adjustment for multiple hypothesis testing, only LVEDV showed a statistically significant improvement (a decrease) in the CorCap-treated group.  Given the constrictive nature of the device, this result is not surprising.  No adjustment was conducted for the remaining twelve endpoints. 

• Secondary endpoints do not show evidence of beneficial cardiac remodeling

Among the twelve unadjusted endpoints, only three structural and two general quality-of-life (QOL) measures showed statistically significantly beneficial results.  Reviewers noted that the QOL and other functional measures showed a low correlation (r<0.35) with improvements in positive structural endpoints such as LVEDV.[30] The FDA emphasized that structural changes without accompanying functional improvements were not meaningful.[13]

In addition, there was no improvement in any of the endpoints that objectively assessed physical function, such as distance walked in six-minutes or lung capacity, even without multiple hypothesis adjustment.[13] Unfortunately, these functional endpoints were also plagued by non-random missing data.   For example, 34% of control patients and 18% of CorCap patients were missing six-minute walk tests at 12 months.

Safety

• Perioperative mortality high in no-MVR arm

Although the only primary endpoint benefit was seen in the no-MVR stratum, the no-MVR arm was also plagued by a higher rate of adverse events. Four out of 51 (7.8%) CorCap patients in the no-MVR stratum died within 30 days of the operation compared to zero deaths out of 50 patients in the no-MVR control group (p<0.05).[13] As noted, there was no demonstrable efficacy in the trial arm that received MVR.  Consequently, the FDA concluded that the “risk-benefit profile [was] not acceptable in either the MVR or No MVR strata.”[13]

• Severe adhesions

Severe adhesions also posed a safety issue in patients who required subsequent reoperations.  One surgeon who had participated in the trial commented that “the heart and great vessels were encased in some of the most dense mediastinal adhesions I have ever encountered…there were almost no tissue planes between the Acorn sock and any of the surrounding structures.”[31] In some cases, the surgeons were unable to free tissues at all “due to the adhesions and thick fibrotic reaction to the Acorn device.”[31] Acorn and FDA anticipated this problem and agreed in the trial protocol to exclude from the trial patients expecting to need proven and life-saving surgeries, such as the coronary artery bypass graft (CABG).[32] This leaves CorCap patients at a distinct disadvantage should they ever require invasive cardiac surgery.

• Long-term heart constriction

The excessive adhesions formed following CorCap placement raise the danger of heart constriction, a condition in which scarring around the heart eventually leads to reduced blood flow and heart failure. Thirty-three percent of echocardiograms from CorCap patients showed evidence of possible heart constriction compared to 13% of control group patients (p<0.0002), although no symptoms from this constriction were reported during the 18 month follow-up period. The FDA cited concerns about the potential for long-term pericardial constriction from CorCap.

Summary of Pivotal Trial Data

The pivotal trial thus yielded data that were marred by large amounts of missing blinded baseline data, relied upon an unblinded primary endpoint component that was the one component of an incoherent endpoint to reach statistical significance, multiple secondary hypotheses that suggested a lack of true structural or functional heart improvement, and safety concerns inherent to the device, such as increased adhesions and possible long-term heart constriction.  The “confirmatory” trial for CorCap would need to show what Dr. Jonathan Sackner-Bernstein of the MDDRP called “a really compelling, irrefutable, internally consistent, and without doubt clinically meaningful benefit,” to overwhelm the concerns raised by the findings from the pivotal trial.[33]

Ongoing “Confirmatory” Trial

Background

This trial is an uncontrolled and unblinded study in which all patients undergo MVR surgery with concomitant CorCap placement.   According to the description provided on clinicaltrials.gov, it is currently enrolling 50 patients in need of MVR with dilated cardiomyopathy, NYHA classes II-IV, and LVEDDi’s between 30 mm/m2 and 40 mm/m2 (the group with claimed positive findings in the “focused cohort”).[7] The patients will be followed for only six months after receiving surgery.  There are five primary outcome measures (all measured at six months unless otherwise indicated): 1) change from baseline in the MLHF; 2) change in exercise tolerance as measured by cardiopulmonary exercise testing, 3) change in exercise tolerance as measure by the six-minute walk test; 4) number of patients who have died or been rehospitalized for heart failure, and 5) peri-operative mortality within 30 days of surgery.  Secondary endpoints are 1) rate of death and adverse events, by cause, at six months; and 2) change in the MLHF at 12 months.  The company anticipates that data for the primary outcome variables will be collected by June 2009. 

The failure of the pivotal trial puts the “confirmatory” trial into context.  Post-hoc data-mining of the pivotal trial data revealed a subset of people within the original randomized population who performed better on the primary endpoint (OR=2.45; p=0.01) and experienced lower perioperative mortality.  But this benefit was not seen in the MVR stratum of the focused cohort (OR=1.81; p=0.16).[34] Thus, the current trial enrolls a subset of patients that has not shown statistically significant improvement in the primary endpoint of the previous trial. The current trial also uses a different set of endpoints, abandoning the previous composite endpoint for five new primary endpoints.  In other words, there are no positive results for the current trial to “confirm.”

Given CorCap’s negative results in the proposed patient population, only a randomized, controlled trial with adequate sample size and follow-up can provide a definitive answer to the question of whether CorCap is a safe and effective intervention for some heart failure patients. Indeed, a drug application with a negative pivotal trial and an uncontrolled follow-up trial would never be approved.

Problems with the “Confirmatory” Trial

• Historical control group

The comparison group that will be used to determine device effectiveness is not reported in the trial’s online summary. The FDA has repeatedly stated that a historical control of some kind may be acceptable for approval.[6] But a historical control that is not concurrent, is not treated at the same centers as the current trial, has not been limited to the same treatments, and may differ substantially in baseline demographics and clinical characteristics is no substitute for a randomized control group.[35] Positive results can result from regression-to-the-mean over time, the placebo effect, as well as the selection of the historical control. Even control patients in the MVR strata of the pivotal trial showed improvements in endpoints for the current trial, such as changes from baseline in the MLHF and six-minute walk.  As a result, distinguishing the true benefit of CorCap from natural improvement will be very difficult. That is why only a concurrent, randomized control group can provide the statistical robustness to interpret positive results from this trial.

• Unblinding and bias

Even if a suitably similar historical control could be found, all fifty patients and their treating physicians in this study will be unblinded.   Again, subjective endpoints like MLHF will be of questionable quality due to possible bias.

• Lack of statistical power

Neither Acorn nor the FDA has revealed what “scaled back statistical criteria”4 will be used to determine CorCap’s effectiveness.  A trial of fifty patients followed for six months is unlikely to generate statistically significant findings, even for the most effective devices.

All of the primary endpoints in the “confirmatory” trial were secondary endpoints in the pivotal trial.  In that trial, there were 193 patients in the MVR stratum, 91 of whom received CorCap.  The FDA concluded that, “In the MVR stratum, the primary and secondary endpoint data collected do not provide evidence of clinically or statistically meaningful changes to support the use of the CorCap in conjunction with mitral valve repair or replacement surgery.”[36] Yet Acorn proposes a trial that will use only about half the number of patients treated in the pivotal trial.

• Unmitigated Safety Issues

Finally, the small size and short duration of this trial mean that it will contribute little additional knowledge about the safety of CorCap. Any major adverse events that might occur will likely be infrequent and thus the study will be statistically underpowered for comparisons with a historical control. Outstanding issues include substantially increased adhesions as well as possible long-term cardiac constriction, neither of which will be satisfactorily addressed in the current trial.

Conclusion

“You could throw it to a patient…but you’re talking more of a humanitarian potential here as opposed to based on a hard, randomized controlled study,” concluded the panelist Dr. John C. Somberg at the original advisory committee meeting in June 2005. The outlook for CorCap does not look much brighter in 2008. 

The low probability that the “confirmatory” trial will produce useful data raises the question of why the trial is being conducted at all. The results of this new trial will be mired in uncertainty unless an unequivocally large benefit is seen. 

Unfortunately, the pivotal trial does not suggest that this will happen. In February 2006, Acorn itself objected that a trial similar to this would not be meaningful.[15]

Approval of CorCap, especially without further advisory committee scrutiny, based on negative data from the pivotal trial with additional data from a non-randomized “confirmatory” trial is likely to be a clear abdication of the FDA’s role to use sound scientific evidence to protect the public.

Yours sincerely,

Eunice Yu
Research Associate

Peter Lurie, M.D., M.P.H.
Deputy Director

Sidney M. Wolfe, M.D.
Director
Health Research Group at Public Citizen