JAMA subjournal: the accuracy rate of randomized clinical trials of real event data replication is less than 20%

Randomized controlled trials (RCTs) are regarded as the gold standard to produce clinical evidence, but real world evidence (RWE) has attracted more and more attention Although RWE can better reflect the patient's demographics, comorbidities, compliance, and concurrent treatment used in the actual clinical environment At present, it is not clear in the industry whether it can be reasonably expected that observation data can be used to solve the same clinical problems answered by traditional clinical trials Recently, JAMA network open published a research result of Dr Joseph S Ross of Yale University School of medicine to explore the feasibility of using real world data (RWD) to copy clinical trial evidence The study examined 220 clinical trials published in 2017 in the seven most influential journals (see chart below); these trials were conducted in the United States The purpose of this study is to explore how many randomized clinical trials can be replicated using RWD in administrative claims or electronic medical records (EHR) Photo source: organized by wechat team of Wuxi apptec Of the 220 clinical trials, 92.7% were randomized trials, 55.4% were double-blind trials, 14.7% were single blind trials and 29.9% were bid opening trials Two thirds (66.8%) of these clinical trials are for testing drug intervention, 15.5% for testing education, procedural or behavioral intervention, 9.1% for testing clinical or surgical procedures, 6.4% for testing medical devices, and 2.3% for testing other interventions The results showed that only 39.1% of the trials used interventions that could be determined from the observation data Researchers cannot determine from EMR data or claims whether the most common interventions are new drugs that have not been approved by FDA (34.1% of the clinical trials investigated), or educational, behavioral or procedural interventions (14.1% of the clinical trials investigated) Of the 86 trials that can be identified for intervention, 39 involve FDA approved drugs, but research on new populations or indications is under way; 32 trials are comparative benefit studies, and 15 are post market application safety or effectiveness studies The results showed that only 33 (15.0%) of the 220 clinical trials analyzed in the United States could be replicated using observation data Specifically, of the 220 trials studied, 86 (39.1%) interventions could be identified from insurance claims and / or EMR data Among the 86 trials, 62 (72.1%) indications can be determined; 45 (72.6%) of the 62 trials can determine at least 80% Inclusion and exclusion criteria Of the 45 trials, 33 (73.3%) identified at least one primary endpoint Photo source: wechat team of Wuxi apptec collated according to reference [1] The results show that we should be cautious about the idea of using RWE to strengthen RCT research Because a lot of data in EMR is not structured data, many experiments can not be repeated The results further indicate that caution is needed before RWE can be considered as a substitute for RCT The author of this paper believes that observational research is not helpful to test the safety and effectiveness of products that are not widely used in clinical practice The author concludes that although there is a potential to put RWD into practice, it is still very difficult The industry still needs to consider the feasibility of using retrospective RWE to supplement the evidence generated by RCT Improving the collection and analysis of EMR data is helpful to improve the use of RWD to generate RWE Relevant improvement measures include increasing the use of patients' reporting outcomes in daily clinical care; wider use of medical device supervision; and improving the natural language processing of free text in electronic medical records "Further development of observational methods and data systems may help to realize the potential of RWE, which in turn translates into broader medical research and faster evaluation of pharmaceutical products." [1] Victoria L Bartlett, Sanket S Dhruva, Nilay D Shah, et al Feasibility of Using Real-World Data to Replicate Clinical Trial Evidence JAMA Network Open 2019;2(10):e1912869 doi:10.1001/jamanetworkopen.2019.12869 [2] Mary Caffrey COTA Collaboration: Helping FDA Figure Out What's Possible and What's Not in Embrace of Real-World Evidence Aug 13, 2019 Retrieved Oct 16, 2019 from https:// [3] Laura Joszt Replicating RCTs With Real-World Data Is Unlikely for Most Trials Oct 11, 2019 Retrieved Oct 12, 2019 from https:// [4] Michael Mezher Researchers Find 15% of Trials Could be Replicated Using Real World Data Oct 10, 2019 Retrieved Oct 12, 2019 from https:// A kind of