Author: Patrick Hughes, Co-Founder & Chief Commercial Officer, CluePoints
We all know that Centralized Statistical Monitoring (CSM) can vastly improve data quality and integrity in a timely manner, boost safety, and slash development timelines. Yet while the benefits of the model have been widely documented, there has, to date, been a lack of information on how to embed it effectively into clinical trials.
Enter the MHRA’s updated Good Clinical Practice (GCP) guidance on Oversight and Monitoring Activities, which includes a specific CluePoints case study to demonstrate the model’s value.
The document, published in January 2022, covers a broad range of clinical trial activities, including the use of committees to manage the trial or review emerging safety data, the central review of clinical trial data, data management processes, and pharmacovigilance signal detection.
Crucially, it also outlines the benefits of statistical monitoring, an aspect of centralized monitoring in which multivariate methods or modeling are used to examine clinical or performance data as it accumulates.
The frequent site-to-site comparisons of clinical and operational data points and Quality Tolerance Levels can flag risk signals by identifying data ‘outliers’, such as unusual patterns, variances, or distributions. This enables quick investigation, allowing study teams to take swift, corrective action before any issues can impact data quality or integrity.
As the MHRA guidance highlights, this sophisticated method can quickly and accurately uncover issues that on-site monitoring can easily miss, such as equipment malfunction, training needs, protocol misinterpretation, rater variability, and even fraud during a trial.
A GCP approach
Some sponsors and CROs have been using statistical monitoring, which can streamline development pathways while protecting participant safety, for some time.
More widespread adoption, however, has been slow. An often-cited reason for this is the lack of clear advice on the practicalities of building the method into routine workflows, but the MHRA GCP guidance helps to address that gap.
Firstly, it talks about roles and responsibilities, explaining that an expansion in the use of centralized monitoring “is likely to develop new roles in this area for statisticians.” CluePoints also sees these roles being defined as Data Scientists or Data Analysts and often originate from the functional area of Data Management. It will also, the document goes on, increase the necessary interaction between monitoring, data management, and other functions with statistics personnel.
In addition, the document says that while many organizations are currently developing quality metrics or key risk/performance indicators for use within statistical monitoring models, there “does not appear to be a list of accepted or validated metrics for sponsors to use.” Some organizations are using multivariate statistical methodologies, such as those provided by CluePoints, enabling the use of several metrics simultaneously, it says.
Some of the most used metrics, the MHRA found, were recruitment rates, screen failure rates, CRF submission/completion times against actual patient’s progress in the trial, query rates, and time to queries resolution vs. the number of active queries. In addition, the number of SAEs reported, missed or late visits/data, and withdrawals/dropouts are also commonly utilized.
The GCP advice recommends that whatever methods and metrics are used that they are documented in the monitoring procedures.
SMART case study
To demonstrate the value of the method, the MHRA has reviewed and shared a case study of the CluePoints SMARTTM engine.
“It is a statistical software solution that examines if data collected are consistent, and if not, pinpoints those investigator sites that differ substantially from the others involved in a trial. In this regard, it is powerful in detecting outlying sites,” says the guidance.
The case study centers on the use of the SMART engine in a phase II vaccine trial of 16,000 patients across 200 investigator sites in the North American, European, and Asian Pacific regions. The statistical monitoring solution performed analyses every six months during the four-year study, comparing multiple data points across the sites.
The multivariate approach meant analyses were performed on all available data, generating thousands of p-values. Then, the scoring algorithm converted these p-values in Data Inconsistency Score (DIS) to pinpoint atypical sites and data subsets.
“All the calculated individual p-values are used to process a score per site and rank all of them accordingly,” said the MHRA, explaining that the more significant the site’s DIS, the more statistically different it is to its counterparts.
“As the chance to identify outliers increases with the number of sites in the study, the SMART engine also offers users an adjustable False Discovery Rate (FDR) that takes multiplicity into account.
“The smaller the FDR, the more confident one can be that a center is truly an outlier. The output is displayed on charts to identify the outlying investigator sites, and this can then be investigated further, perhaps triggering an audit or a site visit.”
Analyses run during the case study trial found several sites in one country (Country X) had significantly lower p-values attached to their body temperature measurements than those in other nations. This prompted an investigation that found all sites in Country X had received miscalibrated thermometers.
The error magnitude was too small to be detected using traditional on-site monitoring but was revealed instantly using CluePoints’ statistical analysis methods. As such, investigators were able to take swift, corrective action, replacing all defective thermometers before the issue could impact data integrity.
Modern trial design
As the case study demonstrates, centralized statistical monitoring (for which CluePoints has become famous) can expedite drug development pathways and protect patient safety. Yet sponsors and CROs have often lacked practical guidance on using the method to gain these significant and long-needed benefits.
Alongside the Cooperative Research and Development Agreement (CRADA) that CluePoints has with the FDA to support their focused site inspections, this represents an essential regulatory message supporting fit-for-purpose technology that enables long overdue process improvement.
The updated MHRA GCP guidance is a huge step forward for modern trial design. Because better clinical trials mean faster access to efficacious, life-changing drugs for the people we, as an industry, all serve.
