Anatomy of a Deceptive Direct to Consumer Robotic Surgery Marketing Campaign – Superior Implant Survivorship
In November 2012, MAKO Surgical Corp. (Fort Lauderdale, FL) added a document to its website titled MAKOplasty Clinical Highlights Survivorship Data 1. The document includes two slides. Slide 1 references a 0.4% 2-year cumulative revision rate (CRR) observed in what is described as a survivorship study 2; the slide is titled RESTORIS MCK Medial Onlay Has 0.4% Revision Rate at 2 Years. Slide 2, titled Robotic arm assisted UKA patients indicate higher satisfaction levels compared to TKA, reports an 89% satisfaction rate of ‘robotic arm assisted UKA patients [sic]’ in this same series.
In October 2012, these and other data were presented at the 25th Annual Congress of The International Society for Technology in Arthroplasty (ISTA 2012); Two year survivorship of robotically guided medial MCK onlay (Roche et al.).3
MAKO Surgical has since designed its first direct to consumer advertising campaign focused specifically on survivorship. The marketing message (i.e., clinically-proven lower 2-year failure rate) is based on a methodologically corrupt analysis and the study results are presented in a manner that is deceptive.4,5 Notably, the results are not corroborated by (any) previously-reported survivorship data, including reports from significantly larger series.
Introduction – The MAKOplasty Marketing Brochure
Slide 1 contains a bar chart (Figure 1) comparing the 0.4% 2-year RESTORIS MCK CRR reported by Roche et al. at ISTA 2012 to historical comparators, none of which are footnoted. The historical comparators include (1) unicompartmental knee arthroplasty (UKA) revision rates reported in national joint arthroplasty registries identified in the slide as All UKA, Swedish Registry, and UK Registry; and (2), UKA revision rates for implants from Biomet UK Ltd [Bridgend, UK] and Zimmer, Inc. [Warsaw, IN] that originate from the group identified under Swedish Registry. The document identifies ‘ZUK’ as the Zimmer Unicompartmental High Flex Knee and ‘Oxford’ as a registered trademark of ‘BioMet [sic] UK Ltd.’ For purposes of this report, we recognize ‘Oxford’ as the Oxford Partial Knee System. We recognize ‘Swedish Registry’ as the Swedish Knee Arthroplasty Register and ‘UK Registry’ as National Joint Registry of England, Wales, and Northern Ireland.
Superior Implant Survivorship is Not an Endpoint
The bar chart in Slide 1 offers the following author conclusion: Improved accuracy of implant placement achieved with robotic arm assisted UKA leads to superior implant survivorship and patient satisfaction.
‘Superior implant survivorship’ is not a recognized endpoint, is not defined by the authors, and is so ambiguous that it requires interpretation (e.g., the conclusion does not identify the inferior comparators). The literal interpretation suggests that a qualitative measure of survivorship was evaluated (e.g., inferior, good, or superior survivorship). The common sense interpretation is that the authors are claiming superiority relative to the comparators.
Despite the absence of an overt RESTORIS MCK superiority claim in the author conclusion (e.g., ‘In a clinical study it was demonstrated that RESTORIS MCK medial onlay has a lower 2-year CRR than Implant X and Implant Y’), the claim is unambiguously communicated in the MAKO Surgical created bar chart, which presents a side-by-side comparison of the RESTORIS MCK unicompartmental medial onlay, the Oxford Partial Knee and the Zimmer High Flex Knee.
Taken at face value, a reader would conclude that MAKO Surgical’s implant is superior to the competitors’ implants, which, as depicted in the methodologically corrupt graphic, appear to have clinically proven 2-year failure rates 9.3 times and 4.8 times greater than the RESTORIS MCK medial onlay. A reader would also be prone to conclude that the Zimmer UKA has a lower (clinically proven) 2-year failure rate than the Oxford Partial Knee.
Nothing in the slide, including the terminology, should be taken at face value; e.g., superiority is not an endpoint, it is a term used to describe a superiority trial.
Figure 1 – MAKO Surgical Historical Comparator Analysis
MAKO Surgical’s Historical 2 Year Comparator Bar Chart is Designed to Mislead
MAKO Surgical’s registry-based superiority analysis is flawed and the superiority conclusion is both erroneous and dishonest. Non-randomized data cannot support promotional or competitive claims. Randomization, a requirement in superiority trials, eliminates systematic differences and ensures comparability between groups. In contrast, registries essentially embrace systematic differences. Other facts not observed by MAKO Surgical include the following: (1) registry data is hypothesis generating – clinical trials test hypotheses (for example, my implant is better than your implant) and (2) registries are observational and measure effectiveness, not efficacy. Although the terms are often used interchangeably, they are distinct.
The term effectiveness, in the context of a joint arthroplasty registry, describes how an implant performs in a general population of patients over time. In contrast, randomized controlled clinical trials have strict and uniformly applied patient inclusion and exclusion criteria. The report by Roche et al. at ISTA 2012 describes the performance of MAKO Surgical’s robotic arm enabled RESTORIS MCK UKA when implanted in patients hand-picked by four financially-interested and highly experienced MAKOplasty surgeons.
Misuse of Registry Data
In a blog post, Biomet’s CEO, Jeffrey Binder, addressed similar registry-based superiority claims made by a Biomet competitor 6. The article, titled The Use and Misuse of Registry Data, highlights the speciousness of the analytical approach. Although the competitor’s marketing tactics are not a topic of debate in this report, it should be noted that the competitor responded by arguing that registry data is ‘aptly suited for inclusion in marketing collateral’ as long as it is not altered and is footnoted (The Journal of Bone & Joint Surgery; Volume 92-A, Number 10, 8/18/2010). Indeed, if the threshold for inclusion of information in marketing collateral is non-alteration and a footnote, then the competitor has won the debate.
It is our view that MAKO Surgical would be better served expending resources not on creating brightly colored bar charts to support dishonest direct to consumer advertising but on closing the clinical evidence gap separating it from its competitors. The multi-faceted research program featured on MAKO Surgical’s website is populated in large part with references to white papers, annual reports, accuracy studies, surgeon editorials, book chapters, and details of future studies 7. Biomet also maintains a web page of clinical highlights 8. In contrast to MAKO Surgical’s web page, Biomet’s web page features survivorship data from actual clinical trials conducted over a period exceeding two decades. Most would argue that the outcome data reported by Roche et al. covering 201 patients is markedly less informative than the outcome data covering 3,529 patients reported on Biomet’s web site.
MAKO Surgical’s Personal Understanding of Superiority
A view held universally by those in the research community, but not by MAKO, is that a claim of superiority requires a p-value of hypothesis test that is statistically significant. MAKO Surgical’s survivorship study has no hypothesis. In addition, superiority is not an endpoint; it is a statistical test used in randomized, controlled superiority trials. Superiority trials test the hypothesis that an experimental intervention is superior to a control on a specified outcome of an interest. Although the survivorship study being conducted by Roche and colleagues is not testing a hypothesis, MAKO Surgical is funding a study that is (Comparison of haptic assisted versus non-assisted unicompartmental knee arthroplasty – ISRCTN77119437).
The hypothesis in this study is that MAKOplasty will result in less variability in mechanical knee alignment at 3-months post-surgery compared to the Oxford Partial Knee. This surrogate, no-risk primary outcome measure is revealing. Although there continues to be no credible evidence that incremental reductions in alignment error of the magnitude afforded by MAKO Surgical’s robot influence functional outcomes or implant survival, the study offers MAKO Surgical an opportunity to add to its library of accuracy data.
If MAKO Surgical has conviction in the hypothesis that the survival rate reported by Roche et al. is statistically significantly lower than the revision rates of registry-based controls, it has the opportunity to test the hypothesis in an unusually small study by orthopedic industry standards. For example, in order to demonstrate a statistically significant difference between the 0.4% CRR reported by Roche et al. and the 3.7% CRR for the Oxford Partial Knee combed from the Swedish Knee Arthroplasty Register, a sample size of less than 300 patients would be required.
Reporting Quality of Two Year Survivorship Study of RESTORIS MCK is Poor
It is not reasonable to hold the authors of the ISTA 2012 abstract to the same reporting standards that an author submitting a manuscript to a medical journal is held, but it is reasonable to hold MAKO Surgical to the reporting and analytical standards employed by the registries from which it has extracted historical comparator data. The historical comparator revision rates that MAKO Surgical isolated from the SKAR are not comparable to its data. SKAR presents survival curves that represent cumulative revision rates; the CRRs describe the percentage of the patients expected to become revised over time.
Survival statistics are used to calculate how long an implant is left unrevised. With increasing observation time, the fraction of deceased patients increases. These patients are not disregarded because they were at risk of becoming revised during their lifetime and are thus allowed to deliver data for the period they lived. The probability for each revision is related to the number of remaining unrevised patients. The sum of all the probabilities is the cumulative risk of revision, which specifies the risk for a surviving patient of becoming revised at a given time. The Swedish Knee Arthroplasty Register – Annual Report 2012 (pg. 10).
In contrast, MAKO Surgical reported a point estimate of the RESTORIS MCK revision rate that was calculated as the number of patients who responded to a 5-question survey divided by the number of patients surveyed. In other words, missing data (patients lost to follow or who died with an implant remaining in place) was disregarded. From a statistical perspective, the point estimate of the revision rate (0.4%) tells us nothing about the true failure rate. Moreover, the method of calculating the point estimate is inconsistent with accepted standards.
In large, long-term orthopedic studies (i.e., 15+ years), the loss to follow up of significant numbers of patients is common. The mean follow up at the time of analysis of MAKO Surgical’s RESTORIS MCK survivorship study was 32 months and involved a small number of patients. Despite this, the magnitude of patients lost to follow up in the survivorship study (approaching 20%) rivals that of decade-long studies.
MAKO Surgical’s marketing document conceals the number of subjects lost to follow up but the principal investigators have reported lost-to-follow up numbers from subsets of this study elsewhere.
- At ORS 2013, Coon & Roche reported that 16% of patients (18 of 116) and 17% of knees (22 of 128) were lost to follow-up.
- Previously, Conditt et al. reported that 19% of patients (29 of 154) and 16% of knees (26 of 160) were lost to follow up.
The data reported by Conditt et al. is anatomically suspicious (i.e., 26 knees and 29 patients) but the two reports allow for a rough estimate of the loss to follow up in the series reported on by Roche et al. at ISTA 2012.
An assumption in survival analysis is that patients go missing at random. However, reviews have shown that patients lost to follow-up in joint replacement studies have significantly worse outcomes than patients who continue to be followed 10,11. Given the unique characteristics of these patients, reliable estimates of survivorship depend on minimal loss to follow up. In addition, because failure rates in UKA and TKA are so low, the failure to capture even a few revisions can lead to unreliable data. For example, if 4 of the 29 knees lost to follow up in the series reported by Conditt et al. were revised, the 2-year CRR would have been 3.7% compared to the reported 0.7% 2-year CRR. There were 4 ‘very dissatisfied’ patients in the series reported by Roche et al. One can reasonably speculate that these 4 patients are at high risk for revision.
MAKO Surgical’s RESTORIS MCK survivorship analysis is not a survival analysis; it is an unreliable substitute involving simple division. The uncertainty (or certainty) of a survival statistic is assessed by constructing a confidence interval (CI). In plain language, a 95% CI allows one to say that there is a 95% chance that the CI contains the true population mean. Fewer patients result in greater uncertainty and a wider interval.
MAKO Surgical’s superiority analysis is an informative example of the power of a confidence interval to prevent erroneous conclusions and rebut deceptive marketing literature.
Uncertainty and The 95% Confidence Interval
The 95% confidence interval (CI) is essential for assessing a survivorship statistic. The CI describes the uncertainty inherent in a point estimate (e.g. the point estimates displayed in MAKO Surgical’s bar chart) and describes a range of values within which we can be sure, based on certain assumptions and at a predetermined level of confidence (e.g., 95%), that the true value lies. If a confidence interval is narrow, the estimate is considered to be more precise. If the interval is wide, then uncertainty is greater. Intervals that are very wide tell us that the point estimate is unreliable.
Figure 2 (below) is the survival curve for the Oxford Partial Knee as reported in the 2012 annual report of the Swedish Knee Arthroplasty Register (pg. 40) 12. The shaded area represents the 95% CI. As noted, MAKO Surgical’s marketing slides do not provide citations; nonetheless, it appears clear that MAKO Surgical isolated the 3.7% 2-year CRR for the Oxford Partial Knee from the survival curve as presented in the 2011 SKAR annual report 12 (in the present example, data from the 2012 SKAR annual report is utilized). The CI for the Oxford Partial Knee (Figure 2) is narrow and the sample size is large (n=1,932).
Figure 2 – CRR for Oxford Partial Knee 2001–2010 (SKAR 2012 Annual Report)
Figure 3 is the survival curve for the Zimmer UKA as reported in the 2012 SKAR annual report; as in Figure 2, the shaded area represents the 95% CI. The width of the interval is extreme relative to the width of the interval for the Oxford Partial Knee; in addition, the sample size (n=315) is a fraction of the sample size for the Oxford Partial Knee. Figure 4 is a tally of primary UKA implants captured between 2001 and 2010 in the SKAR.
Figure 3 - CRR (%) for Zimmer UKA 2001–2010 (SKAR 2012 Annual Report)
The 95% Confidence Interval Doesn’t Lie
In order to demonstrate the canard that MAKO Surgical’s analysis is, we utilize data from the 2012 SKAR annual report (Figure 2 and Figure 3). At four years, statistics tell us that the true population mean (of the CRR) for the Zimmer UKA falls somewhere between 6% and 24% with a prescribed 95% degree of certainty while the true population mean for the Oxford Partial Knee lies somewhere between 4% and 6%. In other words, statistics tell us that there is less certainty in the point estimate for the Zimmer UKA and more certainty in the point estimate for the Oxford Partial Knee; in fact, the width of the CI for the Zimmer UKA suggests that it is should not be relied upon at all. There is a second equally important observation; specifically, the width of the confidence interval for the Zimmer UKA at four years is nearly three times the width of the confidence interval at 2-years, demonstrating that ‘loss to follow up matters’ 10,11. The interpretation of the 0.4% revision rate reported by Roche et al. is not possible due to MAKO Surgical’s analytical sophistry.
Figure 4 - Implants for primary UKA during 2001–2010 (SKAR Annual Report)
No Basis for Confidence In Reported 0.4% RESTORIS MCK Revision Rate
The 95% confidence interval, although not reported by MAKO Surgical, can be estimated; using Rothman’s equation and the effective number at risk (which we estimate based on a report by Coon & Roche 13), the lower bound of the CI is 95.3%. Looked at another way, the lower bound for the 2-year CRR (4.7%) is 12 times greater than the reported 0.4% revision rate. It is important to note that the CI does not reflect the uncertainty of non-random loss to follow up and, as noted previously, the literature suggests that patients lost to follow up in joint replacement studies have significantly worse outcomes. We are unable to perform the same analysis on the data reported at ISTA 2012 because MAKO Surgical’s slide does not report numbers lost to follow up; likewise, we are unable to perform the analysis on the data reported by Conditt et al. because of the discrepancy between the number of patients lost to follow up and the number of knees lost to follow up.
Reported 0.4% Revision Rate Not Corroborated by Any Other Reported Series
Revision Rate 2.5% in 510 Patient Series
Outcomes from 510 patients who underwent MAKOplasty unicompartmental knee arthroplasty between July 2008 and June 2010 have been reported 14. The revision rate in this series was 2.5%, or 6.3 times greater than the revision rate published in the clinical highlights document on MAKO Surgical’s website.
The average follow-up time is not reported. However, the mean time to revision in the two revised knees was 9.6 months and the range was 1 to 19 months. As with the series reported by Roche et al., these 510 patients were operated on by the most experienced MAKOplasty surgeons.
Wake Forest Baptist
In addition to Michael Conditt, Senior Director of Clinical Research at MAKO Surgical, authors of the abstract included surgeons from Wake Forest School of Medicine or Wake Forest University Baptist Medical Center. Wake Forest purchased a RIO system in 2008 and is a high-volume ‘MAKOplasty Center of Excellence’ featured on MAKO Surgical’s website; Economic Case Profiles; ‘Strong MAKOplasty use with 179 cases in first year’. Gary Poehling and Riyaz Jinnah, co-authors of the study and surgeons at Wake Forest, have disclosed that they are MAKO Surgical stockholders and receive consulting fees from MAKO Surgical 15.
An article published in Medscape Today, titled Computers, Robots Win Niche in Knee Replacement, but Clinical Data Haven’t Caught Up, reported that Dr. Riyaz Jinnah, professor of orthopedics and director of orthopedic research at Wake Forest, performs 150 to 200 MAKOplasty procedures per year 16, or 13 to 17 procedures per month; this ‘utilization rate’ is 2.2 to 2.9 times the average utilization rate of a MAKOplasty system.
Revision Rate 4.2% at 6 Months
An abstract presented at The International Society for Technology in Arthroplasty reports selected outcome data from a retrospective review of 135 MAKOplasty patients who underwent ‘uncomplicated 144 MAKO medial unicondylar replacements [sic]’. The study evaluated the impact of residual patellofemoral and lateral osteoarthritis on outcomes, which were measured at six months. Six of 135 patients who underwent a MAKOplasty procedure required revision, translating into a 4.2% revision rate. This revision rate is 10.5 times greater than the 0.4% 2-year revision rate cited in the recent report.
Initial Series of 770 Procedures – 1.7% Revision Rate at Mean Follow Up of 7 Months; 30 Procedures Aborted Intra-operatively
In an initial series of 770 procedures performed by 39 surgeons at 18 hospitals, 10 revisions were reported at a mean follow up of 7 months, for a reported failure rate of 1.3%. Of these 770 cases, 30 had to be converted intra-operatively for various reasons 17.
Revision Rate 5.0% at 10 Months
Data has been reported on a prospective consecutive series of 40 patients who underwent either manual (n=20) or robotic arm assisted medial unicompartmental knee arthroplasty (n=20) 18.
In addition to standard clinical scores, modes of failure were reported. Average follow-up in the manual and robotic group was 12 months and 10 months, respectively. The authors reported the following:
 Five patients in the robotic-assisted arm had persistent tibial pain that resolved in four patients.
 No revisions in the manual onlay implant group.
 One revision for persistent tibial pain in the robotic-assisted inlay group.
Significantly Longer Operative Time, Few Differences in Post-Op Clinical Measures
An addendum from the 13th Annual Insall Scott Kelly Institute Sports Medicine and Total Knee & Hip Course was published in Reconstructive Review 19. Included is an abstract authored by orthopedic surgeons from Ohio Health’s Grant Medical Center and Doctors Hospital including Sharat K. Kusuma, MD and Ray C. Wasielewski, MD. Both Drs. Kusuma and Wasielewski practice at The Bone and Joint Center at Grant Medical Center in Columbus, Ohio.
More Post-Operative Pain in Robotic Group
The abstract, titled Comparison of outcomes of robotic and manually implanted unicompartmental knee arthroplasty, reports on a retrospective review of 30 robotic-arm assisted and 32 manual instrumentation medial unicompartmental knee arthroplasties.
The authors reviewed clinical and radiographic data for matched cohorts who received robotic-arm assisted UKA or standard instrumentation UKA. Among the findings were the following:
 Operative time was significantly longer in the robotic group.
 There were very few differences between groups in postoperative clinical measures.
 A greater number of robotic group patients reported continued medial-sided knee pain.
 Benefits of MAKOplasty May Be Obviated In the Hands of Experienced Surgeon
The authors observed little to no clinical or radiographic differences between the robotic and non-robotic groups and stated that the “purported benefits of robotic UKA may be obviated in the hands of a surgeon with training and experience in manual UKA implantation.” The full text of the abstract is available on the Joint Implant Surgery & Research Foundation’s website.
Leveraging the MAKOplasty Data to Lure Surgeons, Patients and Hospitals
MAKO Surgical intends to monetize the ‘superiority data’ for both patient-directed marketing and for broader commercial purposes (physician recruitment). In November 2012 during an investor conference call 20 MAKO Surgical’s CEO (Maurice Ferre) stated, ‘we’ve got a two-year study that’s out of four surgeons, and there’s a tenfold improvement on survivorship’. On January 9, 2013, during a presentation at a healthcare conference 21, Dr. Ferre stated that he expects that the data will be helpful in ‘luring’ surgeon saying, ‘I think the clinical data that I mentioned about the 0.4% failure rate at 2 years … is kind of key … and it starts luring surgeons into this’.
Dr. Ferre has also described the research as the most important clinical evidence presented by MAKO Surgical at ISTA 2012 20, stating ‘We are not aware of any other unicompartmental implant system that has shown revision rates this low at two-year follow-up’. In addition to these outlets, the data has recently appeared in advertisements in medical journals 22; see MAKOplasty PKR, with the enabling RESTORIS MCK Medial Onlay, has demonstrated a very low revision rate of 0.4% at 2 years.
Peddling clinical refuse to investors is an activity engaged in by many publicly traded healthcare companies; it is predictable and creates little more than theoretical exposure to financial risk. In contrast, the survivorship data reported by Roche et al. has been transformed into deceptive direct-to-consumer marketing literature that is being used not only MAKO Surgical, but also by hospitals and physicians that offer MAKOplasty, to persuade consumers that the procedure ‘offers significant improvement in patient outcomes’, superior ‘implant longevity’ and superior ‘patient satisfaction’.
The RESTORIS MCK Survivorship Study is Tainted
It is reasonable to believe that the marketing of the survivorship data will intensify considering historical marketing tactics that have ranged from disseminating patently false to imaginary data. For example, one of the four principal investigators in the survivorship study (Andrew Pearle) maintains what can be objectively considered an active direct-to-consumer MAKOplasty marketing campaign on his website 23; he is also a frequent contributor to MAKO Surgical’s blog 24 and has contributed articles with the following titles:
 Robotics and Experience: A Winning Combination for Partial Knee Replacement
 10 Things Your Knees Are Trying To Tell You
 Dr. Andrew Pearle | How to Choose between Partial & Total Knee
 Knee Arthritis: A Swelling Epidemic
 A Weighty Issue for Aching Knees
Dr. Pearle’s Personal Blog
On his personal blog, Dr. Pearle tells prospective knee replacement candidates that robotic knee resurfacing, or MAKOplasty, is safer than total knee replacement (see Why Robotic Knee Resurfacing?). 25
The claim is reckless, raises doubt about the surgeon’s objectivity, and is plain evidence of investigator bias.
The safety of MAKO Surgical’s implants and robotic knee resurfacing system (the RIO Robotic Arm Interactive System) was established based on substantial equivalence, a regulatory construct that, in the case of the robot, involved sawbones and cadavers, not humans. The safety of MAKO Surgical’s implants was likewise established based on substantial equivalence. There is no form of evidence that has demonstrated that robotic arm assisted UKA is safer than TKA (or manual UKA). As such, one has to question the reliability of a study whose principal investigator asserts, in what can only be described as direct to consumer marketing puffery, that robotic knee resurfacing (MAKOplasty) is safer than total knee replacement.
The views expressed by this investigator in a more scholarly forum are striking when compared to those expressed on his blog and on MAKO Surgical’s blog. In June 2011, Dr. Pearle was quoted in an article in Becker’s Spine Review 26; excerpts of Dr. Pearle’s commentary are also available on the Hospital for Special Surgery’s website 26.
 Strong, evidence-based studies showing that robotic technology produces better outcomes are lacking.
 Marketing can only take you so far, and now we have to show that the technology is better, says Dr. Pearle.
 These systems cost so much that hospitals tend to create a marketing effort when they buy one of these products. That is a bad thing because it limits the substantive research that needs to be done for robotics.
Dr. Pearle Cites Study Rebutting MAKO Comparator Analysis
Dr. Pearle has also written about the influence of surgeon experience on UKA outcomes 27.
His commentary is illuminating:
 Another major factor in determining … outcome after partial knee replacement is surgeon experience. This is particularly the case for partial knee replacement.
 [A study showed that low volume] surgeons had a revision rate 6-8x that of high volume surgeons!! [Emphasis added by Dr. Pearle]
 I currently perform 150-200 partial knee replacements per year, which is the most of any surgeon in the greater New York area.
The study (abstract) Dr. Pearle cited is equally illuminating inasmuch as it highlights the flaws in MAKO Surgical’s registry-based comparator analysis; 28
The authors evaluated the causes of failure and rates of revision of the Oxford Phase 3 Medial UKA from data recorded in the New Zealand National Joint Registry. The following findings were reported: (1) High volume surgeons had a revision rate of less than 1% over 5 years; (2) Surgeons who performed 8-12 partial knee replacements per year had a revision rate of 5% over 5 years; (3) Surgeons who performed less than 8 partial knees replacements per year had a revision rate of 6-8% over 5 years. In other words, Dr. Pearle seems to agree that registry data is not a reliable tool for evaluating relative efficacy.
MAKO Surgical, Hospitals and Physicians Are Marketing the Superiority Claim
Hospitals and physicians’ groups have begun utilizing and in some cases embellishing the data in direct to consumer marketing campaigns. The lead article in a newsletter that recently became available on South County Hospital Healthcare System’s website is representative. The article depicts the flawed collage of marketing data as legitimate, ‘peer-reviewed’, ‘internationally recognized’ and conclusive evidence of superiority.
Entitled, Good News for Orthopedic Patients – Research Confirms Significant Benefits of Mako Technology, the newsletter cites ‘Recently released clinical research data’ that confirms that ‘Mako technology offers significant improvement in patient outcome for … knee resurfacing’ 29. The author of the article ties the unproven claim of superior outcome to an unproven correlation with implant positioning (i.e., ‘superior implant positioning’). South County fails to cite any studies that have investigated or proven the hypothesis that ‘superior implant positioning’ results in superior patient mobility, superior implant longevity or superior patient satisfaction.
South County Hospital Should Confer with an Expert
At the 2013 Orthopaedic Research Society Annual Meeting (ORS 2013) Mark W. Pagnano, M.D. hosted a workshop that addressed the lack of evidence correlating the precision offered by robotics and navigation with clinically relevant differences in patient function or implant durability 30. Dr. Pagnano is an orthopedic surgeon with the Mayo Clinic [Rochester, MN] and through his affiliation with the Mayo Clinic has a commercial relationship with MAKO Surgical; see Customer Commercial Arrangements – MAKO Surgical Corp31.
A summary of his presentation, entitled Both Computer Navigation and Robotics Improve Precision in Knee and Hip Arthroplasty: But Clinical Benefits Remain Poorly Defined, states the following:
Numerous publications center on the use of image-based haptic robotic systems, particularly in regard to partial knee replacement, and clearly demonstrate reliable and reproducible component positioning. What remains unclear is the effect of that incremental improvement in component position on either short-term clinical function or long-term implant durability. (pg. 14)
Pervasive Conflicts – MAKOplasty Franchisees
It is important to recognize that the 0.4% revision rate reported by Roche et al. was observed in a series of patients enrolled in IRB approved outcomes registries at four of MAKO Surgical’s highest volume customer/hospitals.
Operations were performed by four of the most experienced MAKOplasty surgeons.
Dr. Roche performed the first MAKOplasty procedure on June 14, 2006 at Holy Cross Hospital.
Dr. Coon is a consultant and a designer surgeon for MAKO Surgical. According to an advertisement, The Coon Joint Replacement Institute was ranked ‘number one in the world for MAKO robotic joint replacement volume’ 32.
Andrew Pearle has taken credit for performing more partial knee replacements in the greater New York area than any other surgeon, was the first surgeon at HSS to perform a MAKOplasty procedure, and recently reported that he performs >200 MAKOplasty procedures per year 33.
Frederick Buechel, from Physicians Regional Healthcare System, was previously listed as a principal investigator in the survivorship study 2 and it is not clear whether he is still participating; the ISTA 2012 abstract now lists Jon Dounchis, also from Physicians Regional Healthcare System, as one of the authors. Physicians Regional regularly holds Community Lectures to hawk MAKOplasty. The seminar announcements are titled: MAKOplasty: Why It is Superior to All Other Technology 34.
The Extreme End of the Conflict of Interest Spectrum
It is also important to recognize that all of the principal investigators are ‘interested parties’ with commercial relationships with MAKO Surgical Corp 31. Of the four principal investigators, all are paid consultants for MAKO Surgical. Three of the four surgeons hold MAKO Surgical stock or stock options. Two of the four surgeons receive royalties from MAKO Surgical. 35,36. Financial conflicts are common in all medical specialties and do not necessarily lead to bias. Nonetheless, there are hospitals and surgeons involved in the survivorship study that have what can accurately be characterized as lucrative MAKOplasty franchises that would presumably be negatively impacted if a study were to show ‘inferior implant survivorship’; see, for example, MAKO Surgical Corp. Investor Relations Event: Presentations from Featured Surgeons – AAOS February 8, 2012, San Francisco.
Domestic Joint Replacement Registry
Without accepting the merit of MAKO Surgical’s registry-based comparison, it is worth noting MAKO Surgical’s consistent and transparent optimization of historical controls; for example, Slide 2 of the marketing document compares the reported 89% RESTORIS MCK satisfaction rate to TKA satisfaction rates. MAKO’s marketing department is ostensibly satisfied with the foreign registry comparators it has portrayed on Slide 1, none of which are footnoted, but a domestic joint arthroplasty register is also at MAKO’s disposal.
The Kaiser Permanente (KP) National Total Joint Replacement Registry (TJRR) is a domestic registry designed as a post-market surveillance system for hip and knee replacement 37. It was created to identify best practices, evaluate risk factors associated with revision surgeries, assess the clinical effectiveness of implants, and study newer technologies used in joint replacement. Between the time of its inception in 2001 and July 2012, the registry recorded >148,000 joint replacement procedures. At the 2012 AAOS Annual Meeting, data from an analysis of 1,784 UKA procedures performed between 2001 and 2009 was presented 38. The poster reported revisions recorded in the TJRR for the same historical comparators identified by MAKO Surgical (Table 1), and noted the following: ‘The Zimmer UKA and the Oxford Mobile Bearings [sic] have low revision rates comparable to TKAs in our registry (1.1% and 1.7% respectively).’
Table 1 – Zimmer UKA and Oxford Partial Knee RRs – Kaiser Permanente TJRR
The revision rates depicted in MAKO Surgical’s marketing slide for these historical comparators are greater by a factor of 1.7 times and 2.2 times, respectively. The TJRR data presented above includes 100% of the Zimmer and Oxford knees enrolled in the KP TJRR; in contrast, we estimate that the series reported by Roche et al. is missing data for ~18% of knees.
Kaiser, in its words, utilizes an evidence-based approach when considering what technologies are made available to its members. Data on MAKO Surgical’s RESTORIS line of implants is not captured in the TJRR. Kaiser explains why in its description of its new technology review policy 39:
Our Inter-Regional New Technology Committee examines the effectiveness of new procedures and devices. Our committee decides whether to recommend a new technology based on its findings. Even though a device may have FDA approval and aggressive marketing, the committee may recommend against its use if there’s not good evidence that it’s an improvement over existing technology. For instance, we don’t use . . . MAKOplasty knee replacements [emphasis added]
More Misleading Data – Patient Satisfaction Rates
Slide 2 of the marketing document, entitled Robotic arm assisted UKA patients indicate higher satisfaction levels compared to TKA, compares patient satisfaction levels to those reported in a paper published in Clinical Orthopaedics and Related Research; The John Insall Award: Patient expectations affect satisfaction with total knee arthroplasty 40. Michael Conditt, MAKO Surgical’s Director of Clinical Research, co-authored this paper.
In this study, the authors analyzed factors that contribute to patient satisfaction after TKA; patients completed a self-administered questionnaire that assessed a range of functional outcomes and satisfaction rates. 75% of patients reported being either “satisfied” or “very satisfied” and 14% were “dissatisfied” or “very dissatisfied at 1 year post-TKA.
The Swedish Knee Arthroplasty Register was selected by MAKO Surgical to declare superior survivorship but was not selected to declare superior satisfaction. However, data from the Swedish Knee Arthroplasty Register has been analyzed to evaluate satisfaction rates 41; see Table 2. Satisfaction was based on responses to a four-point questionnaire. The result: A higher percentage of medial UKA patients were satisfied (83%) compared to patients who underwent TKA (82%).
Which Satisfaction Data Set is More Informative?
The 75% TKA satisfaction rate reported by Noble et al. was based on responses from 253 patients. The 82% TKA satisfaction reported by Robertson et al. was based on responses from 12,298 patients. The 89% satisfaction rate reported by Roche et al. was based on responses from 200 patients; the 83% satisfaction rate of medial UKA patients reported by Robertsson et al. was based on responses from 7,680 patients. Other TKA patient satisfaction studies have been reported 42; the study by Noble et al. reports the lowest percentage of satisfied patients; see Table 3.
In addition to this data, high satisfaction rates in patients undergoing unicompartmental knee arthroplasty (medial compartment osteoarthritis only) have been reported43 by Bhattacharya et al. In this study, a fixed-bearing UKA (Preservation, DePuy UK) was compared to a mobile bearing UKA (Oxford); there were 79 patients in the Preservation group and 44 patients in the Oxford group – mean follow up was 5.6 years. The patient satisfaction rates in the Preservation and Oxford groups were 83.5% and 93.9%, respectively. Finally, Rajasekhar reported a 92% satisfaction rate in a study of 135 knees implanted with the Oxford phase 2 prostheses 44.
Table 2 - Satisfaction Rates – TKA v. UKA (Robertsson et al., 2000)
Table 3 – Satisfaction Percentages after TKA
Conclusion: In the past, MAKO Surgical and its customers have demonstrated, although rarely, a measure of restraint by hedging the library of off-the-shelf marketing messages highlighting the benefits that MAKOplasty might, may, or could offer; e.g., less scarring, smaller incisions, reduced blood loss, shorter hospital stays, rapid recovery and ‘restored mobility and a return to an active lifestyle’ 45. The restraint appears to have been abandoned. The survivorship data presented at ISTA 2012 has been adapted into a direct-to-consumer marketing program that portrays the data as evidence of superiority on a critical outcome measure – survivorship – when compared to the Oxford Partial Knee, the Zimmer UKA and a myriad of implants included in foreign knee arthroplasty registries.
The substandard reporting practices, the surgeon impartiality, evidenced by one investigator’s simultaneous role as principal investigator and extremely enthusiastic and compensated pitchman, and the data itself can and should be ignored. What should not be ignored is the patently deceptive direct to consumer marketing campaign that is not supported by the data.
Robotic-Assisted Surgery in the News
Paul Levy recently blogged about direct-to-consumer marketing of robotic surgery systems by Intuitive Surgical and MAKO Surgical’s advertising campaign.
Gary Schwitzer (Health News Watchdog blog) has also written about safety issues associated with robotic surgical systems, questions of evidence and marketing.