CluePoints' Blog

Risk-Based Monitoring Software

Webinar: Risk-Based Monitoring Implementation – What does ICH E6 Rev2 Mean for My Company?

By | Blog, Events, News | 2 Comments

With the official publication of the long awaited revision of the ICH E6 Guidelines, now is the time to take advantage of the new information to ensure appropriate consideration of risk in study design and management.

For this webinar, CluePoints is again teaming up with Paragon Solutions, to highlight the key changes in the mandate and explain how best to address these elements for successful Risk-Based Monitoring Implementation.

Expect to Learn:

  • Understanding the key ICH E6 updates related to Risk-Based Monitoring/Data Quality Oversight and their purpose
  • Understanding how Central Statistical Monitoring(CSM) and Key Risk Indicators(KRIs) will work for you
  • Critical-to-success methods for Central Statistical Monitoring and KRIs
  • Change Management – Using E6 R2 to support your organization’s transition to RBM
  • The Monitoring plan – not just for monitoring intervals anymore


Tuesday, March 14th, 2017, 3:00pm GMT/ 11:00am EDT/ 08:00am PDT (Timing reflects  daylight savings time)

CluePoints – Finalists in The Clinical Informatics News 2017 Best Practices Awards at SCOPE, Jan 24th – 26th in Miami

By | Blog, News | No Comments

We’re thrilled to announce that CluePoints has been named as finalists for the Clinical Informatics News 2017 Best Practices Awards, for our award-winning Risk-Based Monitoring and Data Quality Oversight Solutions. The Clinical Informatics News Best Practices Awards recognize outstanding examples of applied strategic innovation – partnerships, deployments, and collaborations that manifestly improve the clinical trial process.

The winners will be announced at the SCOPE Summit for Clinical Ops Executives in Miami, Florida on January 25, 2017 – wish us luck!

The nomination for this award follows news that CluePoints signed a Cooperative Research and Development Agreement (CRADA) with the FDA. This Cooperative Research and Development Agreement (CRADA) explores a data driven approach to selecting sites which exhibit anomalies indicative of fraud, misconduct or sloppiness.

Also, be sure to stop by our booth (#509) at SCOPE to learn more about Risk-Based Monitoring, Data Quality Oversight and how our agreement with the FDA is driving how the regulator is interrogating submission data to select sites for inspection.

See you there!


CluePoints CEO, Francois Torche, to Present at SCOPE Summit for Clinical Ops Executives

By | Blog, News | No Comments

The 8th Annual SCOPE Summit for Clinical Ops Executives, taking place on January 24-26, 2017 in Miami, FL, will offer three stimulating days of in-depth discussions in 13 different conferences, six pre-conference workshops, and two symposia focused on issues related to each aspect of clinical trial planning and management.

Francois Torche, CEO, CluePoints will lead a session entitled “RBM Demystified: What We’ve Learned from the Practical Implementation of RBM and what the Regulators Want.” The session will take place on Wednesday, January 25th at 11:40 am.

Key objectives for Francois’ session include:

  • Best practice for Risk-Based Monitoring study planning process
  • Key pitfalls to avoid
  • How to differentiate between risk assessment, Key Risk Indicators (KRIs), and Data Quality Oversight Methods
  • How other Sponsors and CROs are leveraging a Risk-Based approach to monitoring and satisfying regulatory requirements

If you haven’t signed up for SCOPE Summit for Clinical Ops Executives, you can save $200 on registration using the following discount code: CLUE

Also, be sure to stop by our booth (#509) to explore the data-driven approach to quality oversight and learn more about how our recent agreement with the FDA is driving how the regulator is interrogating submission data to select sites for inspection

See you there!

Secrets to Successful Risk-Based Monitoring Webinar Poll Results

By | Blog, News, On-demand Webinars | No Comments

Thank you to all who registered and attended CluePoints’ recent webinar with Paragon Solutions: The Secrets of Successful Risk-Based Monitoring Revealed. We hope that you found the webinar informative. Below you will find the poll results and the documented Q&As. The on-demand recording can be viewed here. Enjoy!

The Poll Results

1: How aware is your company of the Risk-Based Monitoring approach to study management?

We are actively planning to implement an RBM strategy


We have announced our plans to move to RBM strategy


We have implemented RBM strategy


We have not assessed RBM approach yet


What is RBM?


2: How many clinical systems are used to manage studies and data today?









3: When you send out communications across your enterprise, what term do you use to describe this approach?

custom coined term


Adaptive Monitoring


Quality by Design (QBD)


Reduced SDV


Risk-Based Monitoring


4: How many KRIs have you defined as standard/core for your enterprise?

5 or less








We have not defined any standard/core KRIs


The Q&A

1: Any data related to studies inspected by USFDA which had Risk-Based Monitoring approach against the traditional monitoring approach?

While there is no public domain data available on the success or failure of submitting an RBM study to the FDA – there is good data on the challenges in the existing model:

  • About 50% of sites fail to reach the planned recruitment targets.*
  • The majority if not all milestones missed in the clinical trials—more than 95% of clinical trials do not end on time and on budget as planned in the first place.*
  • 90% of the studies meet their recruitment goals, but at the expense of mostly twice as much time as originally planned.* The costs attributable to these delays are gigantic.
  • Too many avoidable protocol amendments, with a first amendment implemented even before the very first patient has been enrolled—most of them due to the rush towards “First-Patient-In”.

In addition, an analysis published by TransCelerate in 2014 (Steve Young was a co-author) revealed that – on average – only 1% of site-entered eCRF data gets corrected by the traditional approach of 100% on-site Source Data Verification and Review.**

* Tufts Center for the Study of Drug Development Impact Report “New Research from Tufts Characterizes Effectiveness and Variability of Patient Recruitment and Retention Practices”. 15 (1), January/February 2013.

** Therapeutic Innovation and Regulatory Science “Evaluating Source Data Verification as a Quality Control Measure in Clinical Trials”. November, 2014

2: If you were to use a single term that incorporates QBD and RBM but not use those in particular what do you suggest?

We like the ICH term of Quality Risk Management (QRM) – it is a broader term that encompasses the larger paradigm shift that is needed for this transition.

3: What do you think on implementing RBM in ClinPharm studies? Have companies implemented it in CP studies? (i do not mean FIH studies)

We strongly believe that the QBD and RBM approach should apply to any and all clinical research, including ClinPharm studies.  In particular, the approach of identifying and mitigating risk through study design, study planning and execution is simply the appropriate way to approach clinical research.  However, some of the operational details will look different depending on the study type.  For example, some of the more intensive statistical monitoring methods may not apply to studies of very short duration or with very few sites.  Similarly, the level of on-site monitoring and SDV/SDR will likely vary by study.

Businessman pressing an Team concept button.

CluePoints Partners with Metrics Champion Consortium to Define Metrics and Standards for Centralized Monitoring

By | Blog, News | No Comments

8 November 2016

Cambridge, MA – CluePoints, a leading provider of Centralized Statistical Monitoring (CSM) and Risk-Based Monitoring(RBM) solutions for clinical trials, announces its partnership with the Metrics Champion Consortium (MCC), an association dedicated to the development of standardized performance metrics to improve clinical trials. Drawing on the company’s vast expertise in Risk-Based Monitoring, CluePoints will work with the consortium to help drive collaboration in the development of a standardized framework for Central Monitoring, including identification and definition of what is included, who should be involved, and how best to manage the process.

“We are excited to have the opportunity to lead such an important initiative for the Metrics Champion Consortium” comments Steve Young, Senior Vice President of US Operations at CluePoints. “The industry is finally beginning to move forward to large scale adoption of RBM, yet there remain many questions on the topic of Central Monitoring, and so we look forward to driving clear guidance and standards for this critical component of RBM success.”

The Metrics Champion Consortium has created the Central Monitoring work group to help facilitate the exchange of knowledge and expertise in standardization of clinical risk management. Known for organizing work groups with the intention of providing metrics for the clinical trials industry, MCC has engaged CluePoints to work with them to more clearly define what Central Monitoring is for the industry.

“We are delighted that CluePoints will lead our Central Monitoring work group” comments Linda Sullivan, Co-Founder and President of Metrics Champion Consortium. “Results from our recent Risk-based Approaches to Monitoring Industry Survey show an increasing number of organizations use/plan to use central monitoring approaches in the next 12 months as compared to two years ago. This new MCC work group will bring sponsors, CROs, central & core labs, consultants and IT system vendors together to explore what key risk indicators, performance metrics and data should be included in central monitoring, what analytic approaches should be used to gain valuable insights and how organizations should use the insights to ensure patient safety and improve data quality. CluePoints’ experience with centralized monitoring and risk-based monitoring make the company ideally placed to bring the community together to share insights and align thinking to established, shared global standards.”

For further information on CluePoints’ solutions, please visit

About CluePoints

CluePoints® is a Risk-Based Monitoring and Central Statistical Monitoring solution that has been designed and perfected over the last 10 years. It employs unique statistical algorithms to determine the quality, accuracy, and integrity of clinical trial data both during and after study conduct. Aligned with guidance from the FDA and EMA, CluePoints® is deployed to support traditional on-site monitoring and to drive a risk-based monitoring strategy. The value of using CluePoints® lies in its powerful and timely ability to identify anomalous data and site errors allowing improvement in clinical data quality, optimization of on-site monitoring and a significant reduction in overall regulatory submission risk.

CluePoints Media contact
Patrick Hughes – Chief Commercial Officer, CluePoints
+44 (0) 7703 532 749

About the Metrics Champion Consortium

MCC—a trusted partner in the clinical trials industry—identifies what to measure, how to assess the critical components of what is changing and how the industry is responding to address the changes to make improvements. MCC continually brings you new insights into the leading trends within the industry.

For more information about MCC publications, metric sets & tools, metric education programs and how you can participate in future surveys and MCC work groups, please visit

Linda B. Sullivan – President, Metrics Champion Consortium
+1.317.622.0266 ext 102


CluePoints Introduces Data Quality Oversight Service for Site Inspection Readiness

By | Blog, News, Site Inspection Readiness | No Comments

31 October 2016

Cambridge, MACluePoints, a leading provider of Risk-Based Monitoring (RBM) and Centralized Statistical Monitoring (CSM) solutions for clinical trials, today announced a new Data Quality Oversight service, RADARTM, aimed at helping sponsors achieve a data-driven understanding of risks to an inspection-ready level. As regulators insist that sponsors adopt more comprehensive quality oversight methodologies, this new service looks to move away from antiquated processes to a more rigorous statistical approach to operational quality assurance including preparation for site inspections.

CluePoints’ RADAR™ service harnesses the advanced statistical interrogation of clinical and operational study data to identify investigative sites with a preponderance of anomalous, missing or inconsistent data that may require inspection readiness action. The company’s Data Analysts conduct a comprehensive assessment using the software to identify these higher-risk sites and data which may have escaped detection previously. This rigorous approach is anticipated to benefit sponsor submissions by identifying anomalous sites along with the nature and extent of potential data issues.

“The launch of our ground-breaking RADAR service marks another exciting development in data quality-driven risk analysis. This unsupervised, independent solution will help clinical research organizations to move away from existing time-consuming, approximate and costly procedures” comments Franҫois Torche, CEO, CluePoints. “It analyzes data trends over time to help comply with regulators’ standards, ensuring data quality and ultimately mitigating risks and improving the safety of patients.”

Inspiring confidence throughout the process, this service is an extremely cost-effective alternative to traditional site inspection readiness planning. It provides a report that ranks the most anomalous sites, identifies areas of risk across site and patient data and anticipates where regulators might also target their site inspections. Companies are adopting the RADAR service to use across Clinical Operations, and now in particular for Clinical Quality, Auditing, and Inspection Readiness.  Sponsors can access all these benefits through CluePoints’ outsourced service approach, and alternatively can obtain the CluePoints’ platform for internal, enterprise level adoption.

For further information on CluePoints’ solutions, please visit

About CluePoints
CluePoints® is a Risk-Based Monitoring and Central Statistical Monitoring solution that has been designed and perfected over the last 10 years. It employs unique statistical algorithms to determine the quality, accuracy, and integrity of clinical trial data both during and after study conduct. Aligned with guidance from the FDA and EMA, CluePoints® is deployed to support traditional on-site monitoring and to drive a risk-based monitoring strategy. The value of using CluePoints® lies in its powerful and timely ability to identify anomalous data and site errors allowing improvement in clinical data quality, optimization of on-site monitoring and a significant reduction in overall regulatory submission risk.

Media contact
Patrick Hughes – Chief Commercial Officer, CluePoints
+44 (0) 7703 532 749


FDA Signs Agreement with CluePoints to Explore a Data-Driven Approach to Quality Oversight in Clinical Trials

By | Blog, News, Uncategorized | No Comments

12 October 2016

Cambridge, MA –  FDA and its stakeholders have an interest in assuring the integrity of clinical trial data and the protection of participants during the conduct of clinical research.  Misconduct in clinical research, including, but not limited to the falsification or omission of data in reporting research results, places all subjects in that trial at possible safety risk.  Fraud jeopardizes the reliability of data submitted to FDA, and undermines the Agency’s mission to protect and promote public health. FDA and other regulators rely on whistleblowers and site inspections to detect signs of possible misconduct.

Due to the volume of product submissions, FDA can only inspect a small proportion of clinical trial sites.  The determination of which sites to inspect can involve recommendations by clinical and statistical reviewers, CDER’s risk based site selection tool and FDA inspectors’ judgment and experiences.

This Cooperative Research and Development Agreement (CRADA) explores a data driven approach to selecting sites which exhibit data anomalies indicative of fraud, misconduct or sloppiness.  Under this CRADA, FDA and CluePoints, Inc. will develop and test enhancements to CluePoints existing software to produce an ordered list of “anomalous sites”, i.e. sites whose data are highly inconsistent with data from other sites; explore “moderators of treatment effect”, i.e. factors such as center, region, or country that have a statistically significant impact on the magnitude of treatment effect; add statistical tests and models to those already in the existing software; refine the scoring system used to identify outlying centers; add an exploratory tool to identify moderators of treatment effect; test and implement the software in a high performance computing environment; and develop a user-friendly interface for use by medical reviewers and other interested parties at FDA.

Anticipated benefits to the FDA of the CRADA’s data driven approach include the detection of anomalous sites which may have escaped detection previously, rapid turnaround of results, the ability to determine the nature and extent of data anomalies, and the ability to explore the interaction of various factors with data quality.  These benefits are expected to not only accrue to the site inspection process and improve data quality for all reviewers, but may also inform the efforts of clinical and statistical reviewers to conduct sensitivity analyses, subgroup analyses and site by treatment effect explorations.

For further information on CluePoints, please visit

About CluePoints

CluePoints® is a Central Statistical Monitoring solution that has been designed and perfected over the last 10 years. It employs unique statistical algorithms to determine the quality, accuracy, and integrity of clinical trial data both during and after study conduct. Aligned with guidance from the FDA and EMA, CluePoints® is deployed to support traditional on-site monitoring and to drive a risk-based monitoring strategy. The value of using CluePoints® lies in its powerful and timely ability to identify anomalous data and site errors allowing improvement in clinical data quality, optimization of on-site monitoring and a significant reduction in overall regulatory submission risk.

Media Contact

Patrick Hughes – Chief Commercial Officer, CluePoints
+44 (0) 7703 532 749

Screen Shot 2016-10-10 at 09.27.41

CluePoints Teams up with Paragon Solutions to Share the Secrets of Successful Risk-Based Monitoring Implementation

By | Blog, News | No Comments

CluePoints and Paragon Presents: The Secrets of Successful Risk-Based Monitoring Implementation Revealed

We’re delighted to announce that are our next Risk-Based Monitoring implementation webinar is open for registration! Register now – this is THE Risk-Based Monitoring webinar that you need to attend.

About the Webinar

With regulatory bodies such as the FDA, EMA, and the ICH all strongly influencing sponsors to adopt a risk-based and centralized approach to site and data monitoring, sponsors are often left wondering how to implement this new paradigm most effectively.

For this webinar, Steve Young, Senior Vice President of US Operations, CluePoints,  is teaming up with Karen McCarthy Schau, Principal Consultant, Paragon Solutions, clinical and regulatory experts to the Life Sciences industry, to arm you with all the information you need to plan, document, implement, and evolve a bullet-proof Risk-Based Monitoring strategy.

In this session, we will be laying the foundation for implementing RBM.

Key Takeaways

  • How to structure your organization to support Risk-Based Monitoring
  • How to determine what solutions/capabilities you need to do RBM
  • How to introduce your team to the concept of QBD and risk assessments
  • How to define KRIs and data points to support monitoring

Date and Time

Tuesday November 8th, 2016, 10:00 AM EDT/ 3:00 PM GMT


CluePoints Announces Locations for Risk-Based Monitoring Implementation Roadshows

By | Blog, Events | No Comments

The CluePoints and OmniComm Risk-Based Monitoring Implementation Roadshow is Coming to a City Near You!

In response to the rising complexity and cost of clinical trials, the industry is shifting towards the  adoption of Risk-Based Monitoring (RBM) approaches.  RBM promises to improve both clinical trial efficiency and data quality.  A major challenge to implementing RBM is overcoming perceived complexity and determining roles and responsibilities for all stakeholders.

Explore a day-in-the-life of a team responsible for successfully planning and executing RBM, revealing how you can lead your organization’s RBM journey.

We will review with you the growing evidence in favor of risk-based monitoring, and how to take, what on the surface appears complicated, and make it simple.

Key Takeaways

  • Understand the benefits from an integrated EDC, Targeted SDV, and centralized analytics/KRIs
  • Learn best practices for RBM study planning and execution, and key pitfalls to avoid
  • Draw a clear line of sight between study risk assessment and operational KRIs and quality oversight methods
  • Capitalize on the best use of statistics in central data review
  • Differentiate between KRIs and data quality assessment

Dates and Locations

November, 29, 2016
08:30 AM -11:30 PM
Moltke’s Palace, Dronningens Tvaergade 2, 1302, Copenhagen, Denmark


November, 30, 2016
08:30 AM -12:00 PM
Intercontinental Paris Avenue Marceau, 64 Avenue Marceau, 75008, Paris, France


CluePoints Answers your Risk-Based Monitoring Implementation Questions

By | Blog, News | No Comments

Q1) Are the indicators percentages? E.g., % AEs for the site v study or just number (which could indicate that there are many AEs as there are many subjects)

The units for each KRI may be different.  In the example of AE Rate, we suggest computing a rate of AE’s per patient visit.  So you can think of this as the average number of AE’s per patient per visit – and a site that has a very low average compared to other sites in the study may be at risk of under-reporting AE’s.

Q2) The concept of Risk-Based Monitoring has not been around for some time – how do sites now feel with this approach – last year, we felt it was a very mixed bag – some being receptive, others very against

We are also seeing mixed feedback from sites, along with a fair amount of confusion and misperceptions about what RBM is and what it means for them.  One misperception is that RBM requires sites to scan or fax lots of patient source charts for remote review/SDV.  While some sponsors indeed require sites to do this, we don’t consider this an RBM-specific practice (i.e., RBM => less reliance on SDV, NOT remote SDV).

Q3) Assume some Sponsors have own DM team and may outsource Risk-Based Monitoring – how do CluePoints ensure activities done don’t overlap with traditional DM activities?

CluePoints provides an advanced platform for centralized statistical data monitoring and KRIs (along with RACT and issue workflow support).  We cannot dictate the assignment of roles and responsibilities between a sponsor and CRO.  However, it’s important to note that the CluePoints solution enables very robust statistical methods to detect anomalous patterns in data and operational quality metrics, whereas traditional data mgt reviews (dictionary coding, data listings reviews, programmed edit checks, SAE reconciliation, etc.) are focused on identifying and cleaning discrepant data on a case-by-case basis.  The two approaches should, therefore, be mostly complementary and not overlap.

Q4) Has the CluePoints platform (SMART engine etc.) undergone inspection by any authorities – PMDA. FDA, EMA, etc. – any feedback if so?

CluePoints has not yet undergone a formal regulatory authority inspection.  However, as Francois mentioned during the webinar – stay tuned for news regarding FDA and CluePoints.

Q5) What’s the recommended approach for small studies – single site, Phase I for example – RBM versus traditional SDV, etc

RBM as a methodology is just as valid for small (phase 1) studies as for large, global studies.  The operational methods of monitoring risk will likely be different, however.  In particular, some of the statistical methods used to detect sites with anomalous/unusual patterns of data will not be practical in a single site study.  But doing pre-study risk planning including use of an RACT, along with the use of some operational KRIs, is a good idea even on smaller studies.

Q6) Do you think the costs of software/technological solutions for Risk-Based Monitoring negates the cost savings of reduced on-site monitoring?

The technology investment needed to enable and support effective RBM is minimal compared to the potential cost savings due to reduced SDV and on-site monitoring.  CluePoints has developed an RBM value model that can be used to illustrate the potential savings for any given study design.  Overall we expect a potential savings of up to 12% of overall study budget – and that includes the average cost of RBM technology and incremental resources to perform centralized monitoring activities.

Q7) 1)Does your engine evaluate distribution of data and then suggest the type of statistics to apply? Assuming normality is common, and skews what are called out as outliers in non-Normally distributed data sets. 2) Regarding Agile SDLC, did this include revising how you approach GAMP 5?

  1. SMART detects the type of variable and chooses the test accordingly.
  2. A variable with C < 11 distinct values is considered categorical and analyzed as a set of C binary variables.
  3. Tests assume normality for continuous variables but a log-transformation is systematically applied to get closer to normality. This design choice was made for three distinct reasons: (1) the engine should run automatically in all cases, (2) the test is likely to be informative even under departures from normality, and (3) other tests will detect outliers and other data anomalies that could render the distributions non-normal.

Q2: We perform all the validation activities after completion of the development sprints included in a release. We then apply a classic CSV approach (IQ/OQ/PQ/MQ).

Q8) 1) How can Risk-Based Monitoring be adapted to trials that have a high number of sites with a low number of patients per site (e.g., oncology trials)? 2) Do you have a reference(s) for the metric that < 2% of data change from SDV?

1)  Studies with a very low number of patients per site do indeed present some challenges regarding pro-active issue detection and remediation – specifically because advanced statistical methods generally rely on a certain minimum volume of actual observations/data per site in order to make statistically significant assessments.  Thus the window of opportunity for applying these methods in such studies will be smaller.  However, there is likely still value in applying these methods to detect issues that may otherwise never have been discovered.  Additionally, use of targeted KRIs for which apriori expectations are available (e.g., timeliness of data entry and query responses, screen failure rates, etc.) can be very effective even with small volumes of enrolled patients per site.

2)  Yes – the TransCelerate group published results of an SDV analysis in the DIA Journal in 2014, and CluePoints’ own Steve Young co-authored the article and was intimately involved in the actual design and analysis.  Please contact CluePoints ( if you’d like a copy of the article.

Q9) Nice presentation – Are protocol deviations part of Risk-Based Monitoring?

Assessing rate or volume of PDs at each site is certainly a worthwhile KRI to consider as part of your centralized monitoring.  A number of CluePoints customers are indeed using or planning to use KRIs related to PDs.

Q10) Once critical cross have been identified that need to be 100% SDV’d, how do we determine what noncross are 50% SDV?

There is no absolute guideline or rule available for deciding how much critical and or non-critical data to SDV at each site.  However, some general best practices to consider would include:

  1. a) focusing relatively more SDV on the first patient (or a couple of patients) at each site, and doing much less SDV on subsequent patients.
  2. b)  focus relatively more SDV on critical data vs. non-critical data, and relatively more on AEs/SAEs than on non-critical data.

We would also suggest (similar to TransCelerater recommendation) that overall reliance on SDV be minimized in favor of centralized statistical methods, KRIs, etc..

Q11) Please define the composition of the Central Monitoring Team and how do they work together?

Many organizations are trying to work this out currently and coming to different conclusions re: roles and responsibilities for centralized monitoring.  However, generally, you should consider a “data analyst” type role with responsibility of performing ongoing review of the statistical monitoring results to identify and organize potential issues for review with a larger clinical operations study team.  Appropriate study team reps should decide on which identified issues require follow-up/remediation and actions assigned and then tracked through completion and ultimate issue resolution.

Q12) 1) How does your system support issue management in terms identifying, tracking of the all issues at one place 2) How does your system support overall communication required for risk based monitoring like escalation of risk/issue, etc. to relevant stakeholders and documentation of this communication?

1)  CluePoints includes very robust issue management/tracking workflow support, which allows effective tracking of all identified issues and associated actions through resolution in one place.  All of the information can also be exported into various formats for sharing and/or for eTMF upload at end of study.

2)  CluePoints issue mgt system allows for email alerts to stakeholders who have new/pending actions in the system.  All actions/responses/statuses are retained and audited within the platform and can be exported into reports/documentation.

Q13) 1) What are the general guidelines for determining how much data is necessary to begin to usefully begin to use the central statistical monitoring tools? 2) Do we have any regulator feedback to date on how sponsors are implementing RBM?

It is difficult to set a lower volume of data required before CluePoints can be run successfully, since studies differ from each other in many different ways (number of centers, number, and type of variables, variables repeatedly measured or not, etc.). Based on CluePoints’ experience with studies of various sizes, a general rule of thumb is that the ODIS analysis requires at least ten centers with two patients, but there are exceptions to this rule. In some trials (e.g. orphan indications), the majority of centers have only one patient. In this case, the analyses can still be performed using individual patients instead of centers as the units of analysis. Some tests require a lower limit of more than 50 measurements of a variable for the center to be scored. This requirement translates to a minimum of 51 patients for variables measured only once, but not for repeatedly measured variables (e.g., variables measured once per patient-visit).  Simulations are currently underway with a wide range of study sizes and data issues, in order to inform the lower volume of data required for CluePoints to have adequate sensitivity and specificity.

Q14) you said you need people with a hybrid background in data management and statistics. Traditionally, people with this background do not fully understand the clinical operations perspective to be able to communicate effectively. Would not a person with a clinical background that also has either DM or Stats background be helpful as well?

Yes, agreed – this would be ideal!  So to your point, we could clarify to say the ideal profile for a central monitoring analyst includes clinical data mgt and/or biostatistics.

Q15) Gathering of the KRI – currently, listings are held within other departments and not often in a user-friendly format (cleaning of free text entries +++), how do you collect the data + clean them when applicable in order to produce the output?

CluePoints does not pre-clean discrepant clinical data before assessing it, and this is generally not an issue since what our platform is looking for are unusual or anomalous patterns of data or operational KRIs.

Q16) How are the KRI decided? Do you keep a core listing and adapt to each study depending on the primary endpoint?

The selection of KRIs is up to each organization and/or study team, and ideally, should be guided by pre-study risk planning process.  However, CluePoints team has deep experience in KRI selection and optimization and consult with our customers to ensure they have the best possible approach.  We are also actively working on developing a core library of advanced/optimized KRIs that we will make available to all customers.

Q17) How deep the search for KRI goes? #Queries per patient per site? Average #visit per patient per site? = potential surrogate to efficacy indicator

We believe there will be a core set of KRIs – likely no more than 15 or so – that should be applicable for use in most studies.  CluePoints would be happy to engage you and your organization and offer our expertise in this area.

Q18) How does the calibration of the RACT works? Is the “weight” of each KRI modifiable “live” to assess the output and determine the best ponderation?

The definition of the weight of each question should be performed upfront. When the team answers the questions, there should no more be modifications of the weight as it might introduce bias.

Q19) To Craig: why would it be beneficial to “reduce” the number of autoqueries? Why is it a goal?

In the conversation about RBM, ROI is often brought up. In looking for explicit savings, it is often seen that there may not be a reduction in site visits—rather, site visits are just re-purposed for SDR instead of SDV. So where can some of the explicit cost savings come from? Well, in study startup, taking a risk based approach, it is very possible to have robust edit checks on the critical fields and less or none on fields that aren’t critical or are part of the risk framework. When you reduce edit checks, you reduce autoqueries. When you reduce autoqueries, you reduce cost. This is concluded from Applied Clinical Trials “High Rate of Autoqueries Demonstrates Benefits of EDC,” the article mentions that “it is estimated that the average cost to resolve each query is $53.87 (Medidata Solutions analysis of PICAS® and CROCAS® data).