This post is the second installment of a two-part series on the potential impact of generative AI, particularly ChatGPT, on validation in the life sciences.
The first installment examined the differences between generative AI and traditional AI and how it can be used to create validation deliverables like user requirements. In this final installment, I'll demonstrate how to use ChatGPT to create test scripts from those user requirements and discuss generative AI's potential impact on validation efficiency when combined with a robust validation lifecycle management system (VLMS).
To fully grasp the concepts discussed in this final installment, it is crucial to read (or even reread) part one of this series. It sets the foundation for what we will be exploring further, so be sure to catch up before delving into part two.
Recapping part one, we asked ChatGPT, a powerful generative AI tool, to create user requirements for 21 CFR Part 11. The first requirement it generated was as follows:
Authentication: The system must be capable of ensuring that each user's identity is unique and secure. This could involve the use of complex passwords, biometrics, or two-factor authentication.
Our subsequent use case demonstrates the results of asking ChatGPT to create test scripts from this output.
“Hey ChatGPT, Create Test Scripts for One of the Requirements”
Note: The request is in blue, and the qualification keywords are in yellow.
Part of the art of generative AI is knowing how to ask the right questions, write effective prompts (i.e., select the right keywords), and use quotations to group items together.
In this example, the create test scripts request was focused on “21 CFR Part 11 Authentication ⋯” because it’s encapsulated in quotation marks. If quotation marks had not been used, each word would have been dissected and processed through the neural network (large language model), most likely producing disappointing results. However, because I used quotation marks, the request (i.e., prompt) was able to deliver results that satisfied my needs.
Create test scripts for "21 CFR Part 11" Authentication: The system must be capable of ensuring that each user's identity is unique and secure. This could involve the use of complex passwords, biometrics, or two-factor authentication.
Sure, here are a few example test scripts for testing the "Authentication" requirement of a system designed to meet 21 CFR Part 11 compliance. These scripts focus on ensuring that each user's identity is unique and secure.
**Test Script 1: Unique User Identity**
- Create two different user profiles with the same username and password.
- The system should display an error message stating that the username is already in use.
- Confirm that the second profile cannot be created.
**Test Script 2: Password Complexity**
- Attempt to create a user profile with a simple password (e.g., "password" or "123456").
- The system should display an error message stating that the password does not meet complexity requirements.
- Confirm that the profile cannot be created until a complex password is used.
**Test Script 3: Password Change on First Login**
- Create a new user profile.
- Log in with the new user profile for the first time.
- The system should prompt the user to change their password.
- Confirm that the user cannot proceed until the password is changed.
**Test Script 4: Biometric Authentication**
- Attempt to log in using biometric data (e.g., fingerprint or facial recognition).
- The system should be able to authenticate the user using the biometric data.
- Confirm successful login.
**Test Script 5: Two-Factor Authentication**
- Attempt to log in to a user profile that has two-factor authentication enabled.
- After entering the correct username and password, the system should prompt for the second authentication factor (e.g., a code sent via text or email).
- Enter the correct code to verify the second factor.
- Confirm successful login.
Each of these test scripts is designed to ensure that the system is capable of authenticating user identities in a unique and secure manner, in compliance with 21 CFR Part 11.
Success! Four test scripts were automatically generated from scratch specifically for the 21 CFR Part 11 “authentication” requirement. The problem is the format. The test scripts above would typically be in a tabular format in validation. That’s the following use case.
“Hey ChatGPT, Put Test Scripts in Tabular Format”
When asked to put test scripts in tabular format, ChatGPT created the following within seconds.
Figure 1: A test script in tabular format generated by ChatGPT.
Unfortunately, this was less than ideal, so I issued another request to convert the above into a Microsoft Word table. Here’s where things got exciting! ChatGPT responded with the following:
As a text-based AI model, I'm unable to directly create tables in Microsoft Word. However, I can provide you with text that you can easily format into a table in Word.
Amazing! Although ChatGPT couldn't directly create Microsoft Word tables, it intelligently explained how it could be done. By following its instructions, I was able to make the table below.
Figure 2: Microsoft Word test script created following ChatGPT instructions.
Combining the Power of VLMS with Generative AI
By integrating ChatGPT with a validation lifecycle management system (VLMS), you unlock the potential to fully automate the validation process. Furthermore, by taking the extra step of integrating it with automated test software, you can achieve even greater efficiency. Not only will you be able to perform automated testing, but you can also automate the generation of the validation content (i.e., the deliverables).
Figure 3 below illustrates what a controlled validation deliverable document would look like inside a VLMS with the content generated by ChatGPT.
When connected via API, this entire exercise would take a few minutes (or perhaps a few seconds). Compare that to a manual effort, which can take days, weeks, and even months. Combining generative AI with the power of a VLMS affords you the same result (possibly better) in minutes.
Figure 3: Chat-GPT generated URS managed by the VLMS.
Figure 3 depicts a URS document, but the same concept applies to other validation deliverables. Figures 4 and 5 show the FRS and OQ documents, respectively.
Figure 4: ChatGPT-generated FRS managed by the VLMS.
Figure 5: ChatGPT-generated OQ managed by the VLMS.
One of the major benefits of using a trusted VLMS is the ability to automatically generate and manage the requirements traceability matrix (RTM). Figure 6 below depicts an RTM that has been autogenerated in the VLMS.
Associations between requirements and test steps are clearly indicated with blue checkboxes. Green checkboxes indicate that the test was successfully executed and passed. In the event of a test step failure, a striking red X will immediately catch your attention. Authorized users can ask the system to check for any uncovered test steps or requirements and then automatically generate a report.
Figure 6: Autogenerated requirements traceability matrix.
Generative AI and Data Integrity
As you can see, the potential of generative AI tools like ChatGPT to revolutionize validation efficiency is truly remarkable. By minimizing human involvement, the risk of errors is significantly diminished, resulting in validation deliverables that are more precise and fully compliant with a higher degree of data integrity.
Controls are still in place, especially if using a validated VLMS, which provides functionality not limited to electronic signatures, revision histories, document control, auto numbering, scheduling, and tracking. Furthermore, because everything is in the cloud, authorized users worldwide can access the validation deliverables.
To learn more, contact one of our validation experts.
