How do you validate a system that keeps learning?
Classic validation freezes a system and proves it. Machine learning models change with data. Something has to give, and the industry is deciding what.
AI/Mar 30, 2026/8 min
The entire architecture of computer system validation rests on one assumption so basic it is rarely stated: the system you tested is the system you run. Lock the configuration, control the changes, and the validated state holds. Machine learning breaks this assumption on purpose. A model that improves with data is, by design, not the artifact you tested last quarter.
The honest starting point
Most AI in pharma today does not actually learn in production. Models are trained, frozen, and deployed as static artifacts: a locked model is just software, and existing validation thinking mostly copes. The hard question is the next step, models that retrain on schedule or adapt continuously. For those, a frozen test report describes a system that no longer exists.
Regulators are not asleep here. FDA's work on predetermined change control plans for medical device AI sketches the likely shape: you validate the process that changes the model, not just the model. Define in advance what may change, within what bounds, monitored by which metrics, with what rollback triggers. The validated object stops being a version and becomes an envelope.
What transfers from classic CSV
- Risk assessment still leads. A model suggesting deviations to investigate is not a model releasing batches.
- Data governance becomes the new IQ. Training data lineage, representativeness and versioning are now part of the qualified state.
- Performance monitoring replaces the periodic review as the heartbeat. Drift metrics are the new audit trail review.
- Human oversight is a design control, not a disclaimer. Who can overrule the model, and is that pathway tested?
The discipline that emerges will borrow from CSV's bones, risk first, evidence always, change under control, but its muscles will be statistical: acceptance bands instead of expected results, monitoring dashboards instead of requalification campaigns. The teams that will handle this best are the ones already comfortable saying our control is the process, not the snapshot. Which, if you squint, is what CSA has been teaching all along.
The systems are learning. The validation profession is about to do the same.