Recent Industry guidelines for testing GxP software present an opportunity for regulated companies to improve the efficiency and quality of Computer System Validation (CSV). This article looks at the objectives and elements of an effective test strategy and explores the evolution from traditional testing approaches, that many in industry have got stuck on, to more current and practical guidance from GAMP5 Second Edition and the draft FDA CSA guideline.
Testing software is a compliance requirement in the GxP regulated industries and should be part of a well thought out CSV strategy. Ineffective test strategies can create unnecessary documentation, inefficiencies, and deliver sub-par quality software that either doesn’t work for the intended use or is prone to defects.
Manual testing is time and resource hungry, especially for large configurable business process systems like ERP, QMS, LIMS and MES. Effective testing requires skilled people who understand the business transactions, the supporting computer systems and infrastructure, and the implications for patient safety, product quality and data integrity. Effective strategies ensure test activities, such as authoring test cases and writing reports, are not a barrier to delivering validated software on-time.
The Purpose of Testing
Testing software is critical; it uncovers defects so they can be fixed, and provides objective evidence that software is working as intended in the users’ environment. Testing as part of a risk-based validation strategy ensures that the software’s technical controls are effective to reduce any potential impact to patient safety, product quality, and data integrity.
The regulatory requirements for testing are non-prescriptive and open for interpretation. Statements from EU GMP and PIC/S Annex 11 such as “evidence of appropriate test methods and test scenarios should be demonstrated”, means the decisions and rationale behind what is “appropriate” are left with the regulated company. How this is done, explained, and justified to the regulator requires expertise, and industry guidelines such as GAMP are an excellent reference for the depth and structure of testing evidence and explanation.
Traditional Test Approaches vs. Current Guidelines
ISPE’s GAMP 4 (2001) and GAMP5 (First Edition, 2008) emphasized the need for test cases to be extensively documented, with defined pre-conditions, step by step instructions and expected results. This served a valid purpose to provide clear objective evidence of completed testing, and to maintain test cases that can be accurately repeated for future regression testing.
ISPE’s GAMP5 Second Edition (2022) and the draft CSA guidance from the US FDA (2022), aligned industry guidance with recent developments in technology and Agile software development methodologies better, recommending test strategies use a combination of scripted and unscripted testing.
Unscripted testing offers an alternative approach (ad-hoc testing, error guessing and exploratory testing) to documented testing and vastly reduces the amount of documentation and time required for test case preparation. Test cases are a set of high-level test objectives, or test requirements without the need for step by step instructions, and rely more on the testers’ experience and knowledge. This gives testers flexibility to be creative, experiment with complex use case scenarios, and increase the test coverage to potentially uncover more defects than scripted testing.
Developing a Test Strategy
An effective test strategy incorporates a combination of unscripted and scripted testing and is based on risk assessments to determine how test approaches are applied.
CSA guidelines state that scripted testing is more rigorous and suitable for high risk functions. Although, on its own, scripted testing has limitations due to its restrictive nature and exploratory testing is well suited to uncover unknown defects in high risk functions. A combination of test types should be considered from manual scripted to unscripted and exploratory test approaches, based on the nature of the software itself and the use case. Unscripted testing may be appropriate for all risk levels; the approach to documenting the unscripted test may vary depending on test objectives and level of acceptable risk, from simple test objectives to test cases that list test scenarios, use cases, and failure modes.
In conclusion, Computer System Validation requires a skilled team of testers, and a multi-faceted risk driven test strategy. The shift in industry guidelines now means that regulated companies can efficiently improve software quality by applying pragmatic test approaches that enable testers to spend more time on testing the software rather than on documentation.
At Factorytalk, we specialize in CSV, backed by a proficient team of testers and a comprehensive, risk-driven test strategy. Embracing the latest industry guidelines, we empower regulated companies to enhance software quality efficiently. If you have any inquiries or need guidance on testing procedures, don’t hesitate to reach out to Factorytalk.
