KRIs and Quality Tolerance Limits: Setting Thresholds That Fire

A key risk indicator watches an operational process and tells you a site is behaving unlike its peers. A quality tolerance limit watches a parameter that affects the reliability of the result itself. They are set differently, breached differently, and reported differently, and conflating them is the most common reason a risk dashboard goes unread.

Both are instruments in the wider risk-based quality management plan, and both fail in the same way when the threshold is chosen before anyone decides what happens when it is crossed.

The Difference That Matters

The distinction is not a matter of vocabulary. An indicator is operational and usually sits at site level: enrollment pace, query aging, screen failure rate, deviation counts. It exists because a process could drift, and drift is worth catching early.

A tolerance limit sits at study level and attaches to a critical-to-quality factor. It exists because there is a point beyond which the reliability of the result is in question. ICH E6(R3) expects a small number of these, defined in advance, with breaches and the response to them described in the clinical study report.

The practical consequence is that indicators can be numerous and adjusted as the study runs, while tolerance limits should be few and stable. A study with thirty tolerance limits has misunderstood the instrument.

Where a Threshold Actually Comes From

Software defaults are the wrong source, because a default knows nothing about this protocol or this population. The sequence that produces a defensible threshold starts with a critical factor, moves to an observable in the data, then to a level, then to a response.

For the level itself there are two honest options. Historical data from comparable studies, where it exists and where the comparison is genuine. Or the spread across sites in this study once enough data has accumulated, which sets the indicator relative to peers rather than to an absolute that nobody can justify.

The second option has a consequence teams often miss: it means some indicators cannot be set at study start. Writing a placeholder and reviewing it after a defined number of participants is more honest than inventing a number and defending it later.

The Action Test

The single test that separates a working threshold from a decorative one is whether a defined response is attached before the threshold goes live. Not a general intention to look into it. A named owner, a specific action, and a record of what was found.

Recording the outcome matters even when the signal turns out to be nothing. A screen failure rate that sits far below peers might mean eligibility is being applied loosely, or it might mean the site pre-screens carefully before consenting. Both are plausible. The value of the indicator is that someone asked and wrote down the answer, which is also what an inspector will look for.

Low query rates are the classic example of a signal that reads backwards. A site with almost no queries looks clean, and often is simply entering data so late that the edit checks have not run. That distinction is a data entry lag question before it is a data quality one, and it connects to the design decisions covered in why query volume rises late in a study.

What Happens the First Time One Fires

The first breach is where the design gets tested. If the response is to widen the threshold, the indicator was never a control. If the response is to investigate, act and record, the plan is working as intended.

A breached tolerance limit is a different weight of event. It calls for a study-level assessment: whether the affected data can still support the analysis, whether the protocol or the monitoring plan needs to change, and what is reported. Because a breach is reportable, the definition should be agreed with the statistician before the study opens rather than negotiated after the fact.

Governance is the part that decides whether any of this survives month three. A signal review with a fixed cadence, a standing attendee list that includes the medical monitor and data management, and a short written record of each decision is enough. Without it, indicators are reviewed by whoever has time, and the response varies by reviewer rather than by signal.

Escalation deserves a sentence in the plan rather than an assumption. Most signals are closed by the person who investigates them. A minority need the study team, and a smaller minority need sponsor oversight. Writing down which is which prevents the two failure modes: everything escalating until the meeting is theater, or nothing escalating until an inspector asks who knew.

Keeping the Set Small Enough to Read

Dashboards degrade by accumulation. Each addition seems reasonable and the set slowly becomes unreadable, at which point the monitoring team returns to reading listings and the indicators are ignored.

Two habits keep the set usable. Retire indicators that have never changed a decision, on a defined review cycle. And keep the reasoning visible next to each one, so that a reviewer can see which critical factor it serves. Where the underlying data has to be pulled from several systems at once, the joins are the harder half of the problem, and are covered in which system owns which part of the study. The platform side sits with Clinera CTMS.

References

ICH E6(R3), Good Clinical Practice, which sets the expectation for predefined quality tolerance limits and for reporting breaches in the clinical study report. ICH E8(R1) for critical-to-quality factors, which is where a tolerance limit should trace back to. The FDA guidance on a risk-based approach to monitoring and its 2019 questions and answers. TransCelerate BioPharma has published work on risk indicator selection that is widely used as a starting set.

Current texts are at www.ich.org, www.ema.europa.eu and www.fda.gov.