The Data Behind Risk-Based Monitoring: EDC, CTMS and Site Metrics
Risk-based monitoring programs rarely fail on method. They fail on data: five systems with five identifiers, visit names that do not match, timestamps that mean different things, and refresh rates nobody states. A risk dashboard is only as trustworthy as the joins underneath it, and the joins are the part that has to be designed rather than assumed.
The method itself is well established. What a study is monitoring and why is set out in the risk-based quality management plan, and the indicators that sit on top are covered in setting thresholds that fire. This post is about the layer between them.
Five Feeds, Five Identifiers
Even a modest study draws on five sources. The electronic data capture system holds clinical data, queries and the audit trail. The clinical trial management system holds sites, visits, staff and deviations. ePRO holds completion and timing. Central laboratory and imaging vendors send results on their own accession numbers. Randomization and supply hold allocation and accountability.
Each was selected for its own reasons and each carries its own identifiers. That is normal and not a fault. It becomes a problem only when a monitoring view assumes they line up, which they generally do not until someone makes them.
Where the Joins Break
Five failures account for most of the difficulty, and all five are cheaper to prevent at setup than to repair once data has accumulated.
Site identity is the most common and the most embarrassing, because it surfaces as the same site appearing twice on a dashboard shown to an oversight committee. The fix is a single master site list, mapped to each system's identifier at setup, owned by one function rather than maintained in parallel.
Timestamps are the most consequential. A form completed on the visit date and entered two weeks later carries two dates, and a monitoring view that uses the wrong one produces a signal about data quality when the real finding is entry lag. Recording event date and entry date separately, and being explicit about which one an indicator uses, prevents a whole category of false signals. The same distinction underlies the pattern described in why query volume rises late in a study.
The Metadata Question Underneath
Visit names, form names and window definitions are study metadata, and the question of who owns them decides how much of this work is needed. When each system holds its own copy, every change has to be reconciled in several places and a monitoring view sits downstream of all of them.
When metadata is owned in one place and distributed, the monitoring layer inherits consistency rather than reconstructing it. This is the same ownership question that determines the cost of a mid-study change, set out in handling a mid-study amendment without revalidating the whole build, and the same one that decides how expensive submission mapping becomes, covered in where CDASH to SDTM mapping breaks.
It is worth naming because it is usually treated as a data management concern and resolved after system selection, by which point the answer is fixed.
Freshness Is Part of the Signal
A dashboard that combines data refreshed nightly with data refreshed weekly is not wrong, but it is misleading unless it says so. A monitor comparing two sites has no way to know that one site's laboratory feed is six days behind, and the comparison will look like a finding.
Stating the age of each feed on the view itself is a small piece of interface work that removes a persistent source of wasted investigation. The same applies to known gaps: a feed that has not arrived should read as absent rather than as zero, because zero looks like a result.
Build the Smallest Thing That Answers a Question
The usual failure pattern is a platform selected first, a full integration attempted, and a dashboard delivered in month seven that nobody has needed since month two. The alternative is narrower and slower to look impressive.
Start from one question that the risk assessment says matters. Bring in only the feeds required to answer it. State the freshness. Put it in front of the people who will act on it, and extend only once it is being used. A narrow view that is trusted changes decisions; a wide one that is not trusted gets screenshotted into slides and ignored.
This also has a validation consequence worth planning for. A view that informs a regulated decision sits inside the quality system, and the assurance effort scales with what the view is used for rather than with how many feeds it has. Building narrow keeps that scope honest.
What a Working Layer Looks Like
Three properties, none of them about visual design. A monitor can trace any number on the screen back to the record that produced it. The age of every feed is visible. And a signal carries enough context that the first action is an investigation rather than a request for a different report.
Where this work sits alongside study management and oversight, the system boundaries are set out in which system owns which part of the study. The integration and analytics side is what we build in clinical and R&D AI, with the module stack in Clinera.
References
ICH E6(R3), Good Clinical Practice, for the expectation that systems used in a trial are fit for purpose and that data integrity is maintained across them. The FDA guidance Oversight of Clinical Investigations, A Risk-Based Approach to Monitoring, and its 2019 questions and answers, for centralized monitoring methods. CDISC standards for study metadata and dataset structure. The FDA guidance on computerized systems and electronic records remains the reference point for the audit trail expectations that any monitoring layer inherits.
Current texts are at www.ich.org, www.fda.gov and www.cdisc.org.
Frequently Asked Questions
What data does risk-based monitoring actually need?
At minimum, clinical data and queries from the electronic data capture system, site and visit records from the clinical trial management system, and completion data from ePRO where it is used. Most studies also need central laboratory feeds and randomization or supply data. The difficulty is rarely obtaining any one of them. It is that each arrives on its own identifiers, its own visit naming and its own refresh cycle.
Why do risk dashboards end up unused?
Usually because a number on the screen could not be traced back to a record, so the monitoring team stopped trusting it. Two other causes are common: a dashboard that mixes feeds of different ages without saying so, producing comparisons that look like findings, and a build that attempted every feed at once and arrived months after the question that prompted it had been answered another way.
What is the most common integration problem?
Site identity. The same site carries different identifiers in the electronic data capture system, the clinical trial management system and the laboratory feed, and without a single mapped master list it appears more than once on a dashboard. Timestamps are the more consequential problem: confusing the date an event occurred with the date it was entered generates a steady stream of false data quality signals.
Should we buy an RBM platform or build the views?
It depends on how much of the difficulty is in the joins rather than in the analytics. A platform brings the indicator library and the interface, but it still has to be fed, and the identifier, metadata and refresh work remains. Where a sponsor already has a data layer that resolves those, a narrower analytics layer is often enough. Where it does not, a platform purchase can simply relocate the integration problem.
Does a monitoring dashboard need to be validated?
If it informs a regulated decision, it sits inside the quality system and the assurance effort should be proportionate to what it is used for. A view that triggers an action recorded in the trial master file carries more weight than an exploratory one. Building narrowly helps here: scoping validation to a view that answers one defined question is considerably more tractable than validating a platform that does everything.
Who should own study metadata across the systems?
One function, in one place, distributed to the systems rather than maintained separately in each. Visit names, form names and window definitions are the joins that a monitoring layer depends on, and when each system holds its own copy every change has to be reconciled several times. This is usually treated as a data management detail and settled after system selection, by which point the answer is effectively fixed.
