Authors: Steve Young, Chief Scientific Officer
Sylviane de Viron, Data and Knowledge Manager
In the face of ever-increasing clinical trial complexity, costs, and timelines, centralized monitoring can help sponsors reap the rewards of safer, more efficient drug development.
Key risk indicators (KRIs) are an integral part of this approach, yet, to date, there has been a relative lack of quantitative evidence published on their effectiveness in improving quality.
To address this shortfall, CluePoints conducted a detailed analysis of KRIs from 212 studies conducted via our risk-based quality management (RBQM) software on behalf of 23 sponsors. The results, published in the Therapeutic Innovation & Regulatory Science Journal, demonstrated that most sites could improve data quality through the use of KRIs.1
From SDV to centralized monitoring
Centralized monitoring, a technology-driven alternative to the labor-intensive, error-prone practice of source data verification (SDV), uses data analytics to detect emerging quality-related risks during a clinical trial.
It enables study teams to investigate and take corrective action before an issue can impact patient safety or data quality. This leads to safer, more reliable, and more efficient clinical trials, faster approvals, and fewer delays to market access.
While first encouraged in a 2016 update to the International Committee on Harmonization of Good Clinical Practice (ICH-GCP) guidelines, it was during the height of the COVID-19 pandemic that this method, the basis of RBQM, really took hold.
Authorities, keen to avoid wide-scale study shutdowns in the face of travel restrictions and social distancing regulations, recommended sponsors and CROs replace on-site with the centralized approach.
Evaluating KRIs
KRIs, which, along with statistical data monitoring, is one of the most commonly used tools in centralized monitoring, plays a key role in RBQM.
At the start of a study, the study team defines a set of KRIs to monitor targeted risk areas. Examples might include non-serious AE reporting rate (AERATE) and eCRF visit-to-entry cycle time (V2ECT). Then, for each of these parameters, they define risk thresholds. These can be discrete values or dynamically set based on a statistical comparison of the trend across all sites. If, during routine ongoing centralized monitoring, a site deviates from these thresholds, the system generates a “risk signal,” indicating a potential issue with data quality and prompting the study team to investigate.
Our analyses evaluated the effectiveness of nine commonly used KRIs in 212 studies and included risk signals from 1,676 sites. This was done to evaluate their ability to identify a problem and prompt action to ensure that identified sites are operating within the expected levels of quality.
Quality improvement was observed in the vast majority of cases, including 82.9% of KRIs (1,680/2,027) as measured by statistical score and 81.1% (1,467/1,809) for observed KRI value.
For those risk signals in which improvement was observed, the sites’ KRI statistical scores moved, on average, 66.1% closer to the expected behaviors by the time of risk signal closure. For observed KRI values, the observed level of improvement was 72.4% toward the expected value.
The below example (Figure 1) illustrates the impact of a KRI on one site’s behavior related to eCRF data entry timeliness. A risk signal was generated when the site’s average entry cycle time (V2ECT) reached 32 days. Following intervention by the clinical research associate (CRA) to discuss the delayed data entry with site personnel, the average V2ECT fell to just four days. This was slightly better than the five-day average observed across all sites in the study – and thus represented an improvement of more than 100%.
As regulation and business need to move forward, organizations look for practical ways to embed risk-based approaches into their everyday workflows. Our analysis provides evidence that the effective implementation of KRIs as part of centralized monitoring yields improved quality as aligned with RBQM objectives.
Reference:
- de Viron, S., Trotta, L., Steijn, W., Young, S., & Buyse, M. (2022). Does Central Monitoring Lead to Higher Quality? An Analysis of Key Risk Indicator Outcomes. Therapeutic Innovation & Regulatory Science, 1-9. https://doi.org/10.1007/s43441-022-00470-5
