Monthly Archives

November 2017

Beyond Risk-Based Monitoring: How Intelligent Analytics are Being Used to Drive Value and Support ICH Compliance

The complexity and size of clinical trials has increased dramatically – in parallel with growing costs and mounting regulatory pressures. The introduction of the finalized International Council for Harmonization’s (ICH) addendum to the ICH E6 Guideline for Good Clinical Practice (ICH E6 R2) earlier this year means that organizations across the industry are currently reviewing the update to understand its implications, including assessing the need for revised strategies and operating models. Those that had not previously prioritised a Risk-Based Monitoring (RBM) implementation are now compelled to more aggressively evaluate this paradigm and determine an effective roll-out strategy. Steve Young, Chief Operating Officer at CluePoints, looks at the tremendous benefits that organizations stand to reap by effectively implementing the core principles included in the ICH update and the significant opportunities that the application of intelligent analytics and centralized statistical monitoring (CSM) may present.

Highly publicized safety issues with marketed drugs, a slowing of innovation coupled with patent expirations, and a continual increase in protocol complexity over the past 15 or more years, has meant that the cost and duration of clinical development has steadily increased while profit margins have dwindled. The increasing complexity of trial designs has added significant risk to the operational success of research, both in terms of the ability to attract and retain patients and in generating reliable results that support marketing approvals.

According to the Tufts Center for the Study of Drug Development, the average number of procedures that clinical research patients are subjected to has nearly doubled since 2001. Each patient is required to participate in an average of 163 procedures per trial, while the total amount of data that requires analysis and reporting has also greatly increased. This has made protocols equally demanding for the clinical research clinics that recruit patients and take them through these procedures. They need to be trained on how to execute all the procedures and ensure that all data is collected correctly.

A review of marketing submissions to the US Food and Drug Administration (FDA) between 2000 and 2012 revealed that 32% of all first-cycle review failures – or 16% of submissions overall – were driven by quality issues.1 This means that one out of every six marketing submissions to the FDA were severely impacted by quality issues due to the reliability of the data collected being insufficient. This is a worrying statistic given the tremendous investment in time, effort and money needed to take a new investigational product through clinical development.

As organizations have begun more closely scrutinizing the drivers of cost in clinical trials, site monitoring has come under increasing focus. An analysis by TransCelerate has provided a quantitative assessment of the contribution that the traditional practice of source data verification (SDV) has on data quality. The retrospective analysis, including 1,168 phase I-IV studies across 53 sponsors, indicated that only 1% of electronic case report form (eCRF) data corrections were attributable to SDV.2 The huge amount of cost and investment that the industry has made in this practice has evidently had very little impact.

The challenges highlighted have spurred the industry to rethink the way that it approaches the monitoring of quality in clinical trials. The FDA and the European Medicines Agency (EMA)3,4 both strongly endorse the move to an RBM approach to monitoring. The FDA in particular is not just encouraging the industry to transition to RBM and CSM, it is actively starting to apply some of the core principles in its processes. Currently, the agency is working with CluePoints to identify improved methods for the statistical vetting of clinical study data to assist the identification of appropriate sites to audit during market submissions.

Very much encouraging an RBM approach as well, the ICH E6 (R2) states that, “the sponsor should develop a systematic, prioritized, risk-based approach to monitoring clinical trials. The sponsor may choose on-site monitoring, a combination of on-site and centralized monitoring, or, where justified, centralized monitoring.”5 The sponsor should ensure that all aspects of the trial are operationally feasible and should avoid unnecessary complexity, procedures and data collection. The guidance also recommends that the approach taken must be controlled and systematic for the lifecycle of the study, and sets clear expectations for appropriate monitoring methodologies and documentation, all of which make the case for a RBM approach.

The ICH E6 (R2) guidance indicates that the implementation of RBM, and more specifically using CSM as a core component of clinical trial execution, is recommended due to its ability to provide “additional monitoring capabilities that can complement and reduce the extent and/or frequency of on-site monitoring and help distinguish between reliable data and potentially unreliable data.”5

Both the regulators and Good Clinical Practice (GCP) are now essentially making RBM an imperative, and a GCP compliance issue as well.

At the heart of ICH E6 (R2), it’s not just about RBM, it’s also about QBD. They shouldn’t be thought of as two separate concepts as they both have the same goal – to improve the operational outcomes of clinical research and ensure that the results generated contribute to reliable decision-making. QBD and RBM both call for on-going assessment and mitigation of operational risk. In the case of QBD, this risk assessment is conducted starting with the earliest stages of clinical research design. The purpose is to ensure that clinical studies are designed from the beginning not just with the scientific merits of the research in mind, but with the aim of ensuring feasibility and success. The approach is also focused on the perspective of the patient, along with that of the investigative sites. For example, how burdensome and complex will the study design be for sites to administer and for patients to submit to? There is a desire to ‘trim down’ study designs and make them easier to execute successfully, with the removal of non-core, unnecessary procedures and assessments.

By taking a QBD approach, sponsors are assessing the risk in their study protocol upfront as the clinical research program is being designed, and refining designs to remove any potential operational risks.

Once a study protocol has been developed and is considered stable or final, QBD becomes RBM. Risk assessment is once again performed on the completed design by a cross-functional study team. Remaining operational risks are identified, prioritized, and risk mitigation plans established to guide all downstream operational study management plans. This is the route to a more targeted approach to quality management in clinical trials, and a robust centralized statistical monitoring (CSM) and key risk indicators (KRIs) solution plays a fundamental role.

Centralized monitoring is an important new arm of quality management and oversight in clinical trials that is being added to traditional centralized reviews, such as clinical data management and medical safety reviews. CSM uses statistical methods to identify unexpected or unusual patterns in clinical trial databases to drive better quality outcomes. The approach ideally comprises two complementary components, KRIs and external quality assessment (EQA)/data quality assessment (DQA).

KRIs are pre-designed metrics that are used to identify sites that are deviating significantly from an expected norm. KRIs monitor known operational risks across all sites in a study with these typically being defined during pre-study risk assessment exercises. These may include the rate of protocol deviations, rate of adverse event reporting, timeliness of data entry, and rates of queries or data errors.

CSM employs multiple statistical tests that are executed against all the patient data in a study, with the purpose of detecting unusual data patterns within particular sites, patients or countries that may be indicative of quality issues. Unlike KRIs which are targeted to monitor for specific pre-identified operational risks, EQA/DQA offer a more comprehensive vetting of all study data. If done effectively, this can expose important quality issues that may have not been anticipated during pre-study risk assessments. The combination of KRIs and EQA/DQA provides a very powerful, comprehensive approach to operational quality and risk monitoring.

A further component of successful CSM is to think about medical safety reviews and, particularly, patient profile reports. An advantage of including these in central monitoring processes is that medical safety reviews can be targeted to the patients that are most important to look at. Patients can now be ranked based on the results of statistical vetting, which allows those who have the most interesting and anomalous results to be prioritized. Patient profile reports also provide a robust tool for exploring identified risks down to the patient level within each site.

The introduction of ICH E6 (R2) means that watching and waiting is no longer an option from sponsors and CROs when it comes to RBM. Rather than fearing the change, organizations should see this as an opportunity to embrace improved and more efficient approaches to trial design, conduct, oversight, recording, and reporting, while continuing to ensure human subject protection and the reliability of trial results. An effective RBM centralized monitoring implementation can provide three key dimensions of value. It will make a considerable positive impact on quality outcomes across a business, resulting in more successful marketing submissions and faster time-to-market. It also offers a pragmatic opportunity to review on-site monitoring that can lead to significant and direct savings in the cost of clinical research and clinical trial budgets. Effective centralized monitoring can also result in shorter study timeframes.

The implementation of a successful centralized monitoring plan will help sponsors and CROs manage the shift towards RBM, while achieving compliance with the new regulatory guidelines. By supporting organizations in bringing significant improvements to the quality of their data, and therefore the success rates of their trials, it will be a vital tool in optimizing operational quality monitoring within this exciting new paradigm for clinical research.


The Bigger Picture of ICH E6 R2: Looking Beyond Compliance

Journal for Clinical Studies

Steve Young, Chief Operations Officer, CluePoints

The introduction of ICH E6 (R2) this past year has rendered the implementation of risk-based monitoring principles a matter of GCP compliance.  While clinical research organizations across the industry are now finally compelled to study the new guidance in order to roll out a compliant RBM strategy, many are still not recognizing the incredible opportunity presented by this paradigm shift.  And the failure to understand the compelling benefits of an effective RBM implementation will inevitably result in missed opportunity.  So instead of viewing the updated ICH guidance as an exercise in compliance, sponsors and CROs need to look beyond simple compliance and towards the transformational improvements they can achieve across their clinical development franchise.

What is the ICH E6 R2 Guideline?

The International Council for Harmonization’s (ICH) addendum to the ICH E6 Guideline for Good Clinical Practice (ICH E6 R2) is the first significant update to the GCP guidance in over 20 years.  The motivation for this update is summarized in the Introduction section of the updated guideline: “Since the development of the ICH GCP Guideline, the scale, complexity, and cost of clinical trials have increased …  Advances in use of electronic data recording and reporting facilitate implementation of other approaches. For example, centralized monitoring can now offer a greater advantage, to a broader range of trials than is suggested in the original text. Therefore, this guideline has been amended to encourage implementation of improved and more efficient approaches to clinical trial design, conduct, oversight, recording and reporting while continuing to ensure human subject protection and reliability of trial results.”

A Crisis in Clinical Development

The first sentence of this summary touches on the significant challenges that our industry has increasingly faced over the past 20 years.  Indeed, the complexity of clinical trials – in terms of total number of procedures performed on patients during a study – has risen by more than 50%.  Not only has this contributed to higher costs and longer development times, but the additional burden placed on both patients and investigative sites inevitably adds risk to the quality and operational success of clinical research.  A review of marketing submissions to the FDA between 2000 and 2012 revealed that nearly one-third (32%) of first-cycle review failures (16% of submissions overall) were failed due to quality issues.2 Considering the immense investment in time, effort and money needed to take new investigational products through clinical development, this is a startling statistic.

The SDV Debate

The spiralling cost of clinical trials has brought under renewed scrutiny the drivers of this cost. As the single largest driver of cost after investigative site payments, site monitoring contributes up to one-third of the total cost of clinical research globally. The traditional practice of 100% source data verification (SDV) – never dictated in GCP guidelines – drives at least half of total site monitoring effort and therefore up to 15% of the total cost of clinical research.

While the cost implications of comprehensive SDV are high, some may argue that it is a necessary investment to ensure requisite data quality. However, the alarmingly high rate of quality-related submission failures demonstrates that this practice has not been sufficient. And there is growing evidence to confirm that this exhaustive, manual, on-site review process is not only insufficient but ineffective as well. An analysis conducted in 2014 on clinical data from 1168 clinical trials showed that the practice of 100% SDV drives corrections to only 1.1% of site-entered clinical data on average.3

The Solution is Here

RBM – along with the concept of quality by design (QBD) – have been strongly endorsed not only in the updated ICH GCP Guidance but in related guidance documents issued by FDA and EMA over the past 5 years.  Both QBD and RBM promise to yield higher quality, shorter timelines and greater operational efficiency in clinical research.   QBD and RBM are actually two components of a single paradigm, as both necessitate ongoing assessment and mitigation of operational risk. QBD is conducted at the earliest stages of clinical research design to ensure that studies are optimized not just for scientific merit, but for operational success as well. The concepts of patient-centricity and site-centricity play important roles in this regard.  Actively considering the perspective (and plight) of the patient and investigator will lead to study designs that are much more acceptable to both, which should improve enrolment, retention and overall compliance.   Once a study protocol has been developed, QBD becomes RBM. Risk assessment is performed on completed designs by a cross-functional study team. Remaining operational risks are identified and prioritised, and risk mitigation and risk monitoring recommendations are established to guide all downstream operational study management plans.

Centralized Statistical Monitoring

ICH E6 (R2) advocates centralized statistical monitoring (CSM) as a core component of operational risk detection, noting that it provides “additional monitoring capabilities that can complement and reduce the extent and/or frequency of on-site monitoring and help distinguish between reliable data and potentially unreliable data.”3 CSM is thus positioned as a key to the operational success of any RBM implementation, and for effective oversight of quality in general.  CSM uses statistical methods to identify unexpected or unusual patterns in clinical data, and is ideally comprised of at least the following three components:

  1. Statistical Data Monitoring (SDM): This should comprise a well-designed, robust set of statistical tests to be run on all of the clinical data in the study, with the purpose of identifying atypical data patterns that may represent operational risks of various types including fraud, study equipment malfunction, site sloppiness and training issues.  SDM as defined here has been very effective at identifying risks that may not have been considered during pre-study risk planning.
  2. Key Risk Indicators (KRIs):  KRIs represent a set of metrics designed to help monitor for known operational risks across all sites in a study.  A few examples of commonly-used KRIs include:
  • The rate of protocol deviations
  • The rate of adverse event reporting
  • Timeliness of data entry
  • Rates of queries or data errors
  • Screen Failure rate and Early Termination rate
  • Rate of missed procedures – especially key efficacy or safety procedures
  • Quality Tolerance Limits (QTLs): Similar to KRIs, QTLs represent metrics designed to monitor for specific operational risks.  However, the focus is on more systematic issues which, according to ICH E6 (R2), “can impact subject safety or reliability of trial results”.  While consensus is still developing on the appropriate interpretation of this new ICH language, QTLs should generally be thought of as monitoring for specific thresholds beyond which the study would likely be considered an operational failure.

The combination of SDM, KRIs, and QTLs can provide for a very powerful, comprehensive approach to operational quality and risk monitoring.  When designed and implemented effectively, CSM not only drives significantly better quality outcomes, but does so with much greater operational resource efficiency – enabling a significant reduction in the reliance on SDV and related on-monitoring reviews.

Effective CSM does not come automatically.  Statistical tests and KRIs that are designed carelessly may lead to a relative inability to identify risks in a timely fashion, and/or a high rate of false risk signalling.  This latter issue has too often resulted in unnecessary risk remediation activities which run counter to the actual intent; i.e., more efficient, targeted quality management.


Today, many organisations are still in the process of interpreting the ICH E6 (R2) Guideline to translate the recommendations into tangible operating practices. A risk-based approach to clinical trial management is now a GCP expectation. And while compliance is a legitimate motivator, the principles included should deliver to your organization much more value than simple compliance:

  • Significant reduction in the cost of clinical development, primarily due to the reduced reliance on 100% SDV and frequent on-site monitoring visits.
  • Shorter study timelines – driven by improved enrolment and retention rates, as well as more efficient database lock processes.
  • Higher marketing approval rates, driven by significantly higher study and data quality.

These should not be considered vague, theoretical or uncertain value propositions.  Organizations are already reporting significant cost efficiencies with roll-out of RBM, and many organizations including those belonging to the TransCelerate consortium are observing significant improvements in key quality measures on RBM studies.  Now is the time to explore how a change in mind-set from simply ensuring compliance towards embracing effective RBM can offer these tremendous business opportunities. The new update has the potential to fundamentally alter how clinical research is managed. Risk-based trial design and quality management will, undoubtedly, be an essential component of the future clinical research landscape for decades to come.


  1. Classification and analysis of the GCP inspection findings of GCP inspections conducted at the request of the CHMP, 1 December 2014
  3. Evaluating Source Data Verification as a Quality Control Measure in Clinical Trials, Therapeutic Innovation & Regulatory Science 2014, Vol. 48(6) 671-680

Author bio

Steve Young holds a Masters degree in Mathematics from Villanova University. He worked for three bio-pharmaceutical companies over a span of 15 years, where he assumed leadership positions in clinical data management and led the successful enterprise roll-out of EDC at both J&J and Centocor. He spent an additional six years with Medidata and then OmniComm, leading the development of analytics and risk-based monitoring (RBM) solution portfolios and providing RBM and clinical operations consulting to many organisations. He also led a pivotal RBM-related analysis in collaboration with TransCelerate, and has co-authored two patents related to RBM methods.

CluePoints Builds on Late Phase work in Japan and Launches Risk-Based Monitoring Roadshow in Tokyo

Wayne, PA – CluePoints, a leading provider of Risk-Based Monitoring (RBM) and Data Quality Oversight solutions for clinical trials, has further expanded its Late Phase work in Japan by entering into two new partnerships with global top 20 pharmaceutical companies. The collaborations mark the latest success for CluePoints in the region, with its centralized statistical monitoring (CSM) platform having already been used by the Nagoya University School of Medicine for an eight-year Phase III Stomach Cancer Adjuvant Multi-Institutional Group Trial (SAMIT).

The new studies will see both sponsors using CluePoints’ solution to check data quality and integrity across all investigative sites and patients. By using advanced statistical methodologies to identify any anomalies or outliers in data, the platform works to target monitoring to the sites and patients where it is most needed. The CluePoints’ solution was used during the SAMIT trial once the committee had recognized that traditional approaches to quality control may not be fully effective. It provided reassurances that the clinical trial protocol had been consistently followed across participating centres.

“Our continued growth in Japan builds on CluePoints’ commitment to becoming the ‘go-to’ Risk-Based Monitoring (RBM) technology in this key market for both regional and global clinical trials,” comments François Torche, CEO, CluePoints. “We are already undertaking analyses of Phase III studies in Japan with mid-tier pharmaceutical companies, improving clinical data quality and contributing to the reduction in the overall risk associated with sponsors’ regulatory submissions.”

Keen to build on its presence in the region, CluePoints will host its next RBM Roadshow in Tokyo in November. The event will explore a day-in-the-life of a team responsible for successfully planning and executing RBM, review the growing evidence in favour of the approach and reveal how delegates can lead their organizations’ RBM journey. The event will take place at the Tokyo Belle Salle, Sumitomofudousan Chiyoda on November 9, 2017, from 1:30 – 4:30pm. For more information and to register, please visit

In support of the company’s wider long-term growth strategy across the Asia-Pacific region, CluePoints is undertaking ‘knowledge transfer’ initiatives on behalf of large pharmaceutical companies in both China and Singapore to equip teams of central data scientists to use its software to analyze hundreds of clinical trials. This will drive significant cost and efficiency savings and increase data quality and integrity in line with the new ICH E6 R2 guidance.

About CluePoints

CluePoints® is a Risk-Based Monitoring and Central Statistical Monitoring solution that has been designed and perfected over the last 15 years. It employs unique statistical algorithms to determine the quality, accuracy, and integrity of clinical trial data both during and after study conduct. Aligned with guidance from the ICH, FDA and EMA, CluePoints® is deployed to support traditional on-site monitoring, medical review and to drive a Risk-Based Monitoring strategy. The value of using CluePoints® lies in its powerful and timely ability to identify anomalous data and site errors allowing improvement in clinical data quality, optimization of on-site monitoring and a significant reduction in overall regulatory submission risk.

Media Contact:

Patrick Hughes – Chief Commercial Officer, CluePoints
+44 (0) 7703 532 749