According to FDA 820.3(z), “Validation means confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled.”
Ironically, the actual provision of objective evidence is subjective. In this post, we’ll examine ways to overcome the subjectivity that can influence the quality of your objective evidence and jeopardize compliance.
First, let’s consider the difference between the words "subjective" and "objective." A subjective perspective is influenced by emotion or opinion and cannot be proved or disproved. Conversely, an objective perspective is factual, measurable, and something we can quantify.
But what exactly is objective evidence? In Clause 3.8.3 of ISO 9000:2015, the International Standards Organization defines objective evidence as “data supporting the existence or verity of something” and notes that “objective evidence can be collected by performing observations, measurements, tests, or using other suitable methods.” In other words, it’s physical evidence (e.g., records, documentation) relevant to the audit criteria that auditors can inspect and evaluate for themselves.
Objective Evidence in Computer System Validation
In the context of computer system validation (CSV), “provision of objective evidence” means software requirements must be identified, and all validation efforts and test results should be documented. “Consistently fulfilled” means the test must ensure that the software will function in all possible scenarios.
Objective evidence should detail what actually happened (or didn’t happen), including what the tester actually observed during testing. Avoid subjective conclusions like “yes/no” and try to minimize the opportunity for confirmation bias. This is easier said than done.
The Dangers of Confirmation Bias in Test Protocols
As humans, we’re prone to confirmation bias, which, according to TechTarget, is “a type of mistake that occurs in thinking when information that confirms a preexisting belief is given priority over information that does not support a preexisting belief.”
Our beliefs can unintentionally alter our observations. Natural bias can creep into our test protocols. This isn’t good because the quality of the objective data we receive is only as good as the quality of the protocols we write. Interpretation of information can also hold bias.
Will Your Objective Evidence Hold Up to Auditor Scrutiny?
When it comes to providing objective evidence to support that a validation activity is, in fact, valid, we are influenced by opinion (expert opinion, if you please) as to what objective is required. We don’t really know if our objective evidence is sufficient until it is scrutinized during an audit or inspection. Then the auditor's or inspector’s professional opinion is subjectively exercised to assess the objective evidence.
Here’s the dilemma: How do we deliver objective evidence with a high degree of certainty that it will withstand regulatory scrutiny?
Applying a scientific method that is risk-based, consistent, and reliable is one way — possibly the only way. But how do we deliver upon these criteria if we depend on manual, paper-based processes that are difficult to manage and maintain and subject to human error? The answer: leverage the power of technology that a validation lifecycle management system (VLMS) provides.
Tackling Subjectivity with Technology
A VLMS replaces paper-based manual processes with paperless electronic records, electronic signatures, allowing users to enforce compliance through programmatic controls, thus minimizing or eliminating the unconscious human bias that compromises compliance.
ValGenesis VLMS can also apply risk-based processes consistently. For example, high-risk requirements can trigger actions that require more objective evidence, such as step-by-step electronic signatures, screenshots, or supporting file attachments at the test step level. This can be captured and maintained precisely where the requirement exists in the form of a hyperlink that, when clicked, will immediately reveal the supporting objective evidence.
Capture and Document Objective Evidence Automatically
If you’ve manually captured screenshots and assembled them in paper documents — the archaic way of performing validation — then you know how tedious this can be. It's also error-prone. With a VLMS, objective evidence in the form of screenshots and file attachments can be captured and documented automatically, directly in the protocol, saving an enormous amount of time and effort.
ValGenesis VLMS can be configured, based on customer-specific requirements, to enforce an organization’s own risk-based methodology consistently. When authorized users perform validation, they must follow the rules; otherwise, they cannot continue. Automation allows for objective evidence to be captured and documented at the push of a button. Also, automated trace matrices illustrate test coverage along with successes and failures. Finally, any failures are required to be addressed successfully through to completion.
Aligned with CSA to Reduce Over-Testing
The life sciences industry’s increasing adoption of the CSA methodology for validation and soft encouragement from regulatory agencies have not escaped the attention of ValGenesis product designers. CSA encourages the use of critical thinking, unscripted test methods (such as ad hoc testing), and automated technologies.
The ValGenesis VLMS is the first-to-market CSA-ready solution that provides users with a seamless way to assign a test category along with a pre-configured list of deliverables for the assigned test category to optimize testing efforts. Users can demonstrate 21 CFR Part 11 and Annex 11 compliance with strong objective evidence and secure audit trails that stand up to strong regulatory scrutiny.
Our system is not only aligned with the FDA’s CSA initiative but also expedites an organization’s ability to adopt and deploy it. Contact our experts to learn more.
