Michelle Grienauer

by Michelle Grienauer

Using a Risk-Based Approach to System Validation

The FDA’s June 21 new draft guidance on 21 CFR Part 11 (Part 11) and the use of electronic records and electronic signatures in clinical investigations (Guidance) is an important document for researchers. It builds upon the approach and recommendations presented in the current Part 11 guidance document, while providing much-needed clarification and detail regarding the use of a risk-based approach to validation, audit trails, and record retention. It also delves into the applicability of Part 11 to different electronic systems, mobile technologies, and telecommunication systems.

 

What is Validation?

The FDA defines validation as the “confirmation by examination and provision of objective evidence that the particular requirements for a specific intended use can be consistently fulfilled.” In other words, validation is the process of proving that a system or software product does what it is intended to do, and will be able to do so throughout the life cycle of the product. A validation plan usually includes documentation of all system requirements, testing against each requirement, and maintaining a change control process.

Unfortunately, faulty software has been responsible for, or created risk of, deaths and serious injuries. Software issues in an electronic trial data system can also bring into question the reliability of study data. Thus, in order to protect the end-user (research participant and/or patient), Part 11 and various other FDA regulations (known as “predicate rules”) require validation of computer systems or software used to fulfill a regulatory requirement.

 

A Risk-Based Approach to Validation – Annex 11 and GCP

One of the criticisms often launched at Part 11 is that, unlike its European counterpart, Annex 11, the text of Part 11 does not mention or promote the establishment of a risk management program and a risk-based approach to system controls such as validation.

For example, Annex 11 begins by stating that risk management should be applied throughout the life of a computerized system and that the decision and extent of validation and data integrity controls should be based on a risk assessment that takes into account the impact such a system has on patient safety, data integrity, and product quality.

In addition to Annex 11, the recent addendum to the ICH Guideline on Good Clinical Practice (ICH-GCP E6 (R2)) promotes sponsor implementation of a clinical trial quality management system that is proportionate to risk. Regarding the validation of electronic trial data systems, ICH-GCP E6 (R2) states that the approach to validation should be based on a risk assessment that considers the intended use of the system and the potential of the system to affect human subject protection and reliability of trial results.

It is important to keep in mind, however, that the lack of mention of risk and quality management programs is a consequence of the technological landscape between 1997, when Part 11 was first published, and 2003, when the current guidance document was published. In contrast, Annex 11 was published in 2011 and ICH-GCP E6 (R2) was published in 2016.

Thus, one of the main purposes behind publishing the new Guidance is to acknowledge the technological advances that have occurred in the last two decades and to encourage and further clarify the risk-based approach to validation of electronic systems.

 

When to Validate

In its new Guidance, FDA acknowledges that a broad variety of electronic records and electronic systems are used in clinical investigations. Thus, the agency states that, when using a risk-based approach for the validation of such systems, sponsors and other regulated entities (e.g. investigators, CROs, IRBs) should consider:

  • The purpose and significance of the record, including the extent of error that can be tolerated without compromising the reliability and utility of the record for its regulatory purpose.
  • The attributes and intended use of the electronic system used to produce the record.

Even more specifically, the FDA states that electronic systems should be validated if those systems process critical records that are submitted to the FDA. “Processing” includes actions such as creating, modifying, maintaining, archiving, retrieving, or transmitting. Examples of critical records include laboratory and study endpoint data, information about serious adverse events and study participant deaths, and information on drug and device accountability and administration.

 

Extent of Validation

The extent of validation should be tailored to the nature of the system, its intended use, and the potential of the system to affect product quality, safety, and record integrity. The Guidance provides examples of the recommended extent of validation for three different classes of electronic systems:

TypeExtent of Validation
- Commercial off-the-shelf (COTS) office utilities software (e.g. word processing, spreadsheets and PDF tools)
- Electronic systems that do not process critical records
- May not be necessary. Extent determined by an organization’s internal business practices and needs.
- COTS systems performing functions beyond office utilities (e.g. COTS electronic data capture (EDC) tools)- Obtain documentation from vendor that includes a description of standard operating procedures and the results of testing and validation.
- COTS systems integrated with other systems
- Customized systems developed to meet the user’s unique business need (e.g. integration with other software systems)
- Sponsor and other regulated entity should develop and document a validation plan; conduct validation in accordance with the plan; and document validation results.
- Validation and testing should be performed before use of the system. Re-validation may also be necessary before implementing changes to the system.
- Vendor audit may be warranted.

 

Who is responsible for validation?

The Guidance makes clear that, even when outsourcing electronic services (e.g. using a vendor’s electronic clinical trial management system), the sponsor and other regulated entities are still responsible for meeting the regulatory requirements and ensuring the outsourced electronic service is validated as appropriate. Additionally, the FDA states that sponsors are ultimately responsible for assessing the authenticity and reliability of all study records and study data. Thus, any potential vendor should be thoroughly vetted in order to assess its ability to meet Part 11 requirements and data security safeguards.

Depending on the type of system and the level of risk involved, this vetting may include conducting a vendor audit. Such audits would assess the vendor’s design and development methodologies and validation documentation. As part of an entity’s due diligence, the FDA also recommends that service agreements are obtained from such vendors that include a clear description of the vendor’s requirements and responsibilities.

Finally, in order to demonstrate the vendor is providing service that comply with Part 11, sponsors and regulated entities should keep the following documentation on site and available to the FDA:

  • Specified requirements of the outsourced electronic service
  • Service agreement defining what is expected from the vendor
  • Procedures for the vendor to notify the sponsor/regulated entity of changes and incidents with the service

While the FDA emphasizes that a regulated entity cannot outsource its regulatory obligations, the Guidance does not explain how to apportion responsibility between the various actors participating in the same research (e.g. sponsor, CRO, site).

Given that the current clinical trial landscape includes a mix of both site-initiated and sponsor-mandated use of outsourced electronic services, there is likely to be some confusion and debate about who has the obligation, or how to share the obligation, of performing actions such as conducting vendor audits, executing service agreements, and performing system validation.

 

Conclusion

The new draft Guidance is a significant move to modernize the agency’s interpretation of Part 11 and explicitly endorse and expand upon the agency’s approach to risk-based validation. However, due to the lack of guidance regarding the appropriate allocation of the burden of due diligence, the industry will likely continue to wrestle with who has the obligation to take on validation. Kinetiq will continue to report on industry changes that improve the outlook of this question. And as always, our expert regulatory consultants are available to work with your organization to negotiate and document the right allocation of due diligence for your project.

The draft Guidance is currently open for comments.

 


Mitchell Parrish

Executive Insight

Mitchell Parrish, JD, RAC, CIP, VP of Legal and Regulatory Affairs

 

Until the FDA updates its Part 11 regulations governing electronic records and electronic signatures, the Guidance is helpful in encouraging more relevant practices. These practices, namely risk-based approach to system controls, functioning risk management programs, and vendor qualification programs, are key in keeping with modern times and in reducing liability exposure.

Tags: , , , , , , , ,