Quality metrics are critical in the measurement and assessment of the quality systems in the pharmaceutical and biologics industry. These quality metrics are used throughout the industry to monitor quality control systems and processes and drive continuous improvement efforts in drug manufacturing.
In 2016 the Food and Drug Administration issued a guidance document titled Submission of Quality Metrics Data Guidance for Industry.
In this guidance document the FDA communicates the importance of Quality Metrics to help develop compliance, inspection policies and practices. The FDA also communicates the importance of quality metrics and their intent to implement a Quality Metrics program.
Validation is a critical indicator of the quality of processes and system during their lifecycle.
Validation metrics can be an excellent tool to monitor the health of critical GMP processes and systems. Metrics also provide data that measures the quality of the process during the lifecycle.
Unfortunately, validation metrics are often overlooked not well understood and defined in the industry. This article will discuss the importance of validation metrics and the relationship to the FDA initiative. This article will also discuss how to identify and implement quality and productivity metrics for validation.
CHALLENGES IMPACTING VALIDATION METRICS
Companies in the industry continue to struggle to identify and implement meaningful metrics that assess the quality of the validation program.
Some of the challenges are related to the fact that companies normally put more emphasis in traditional metrics. Traditional metrics normally involve a combination of quality and productivity metrics normally impacting core processes in the quality system such as deviations, CAPA, lot release and trending of deviations
Most companies implement metrics such as change control, deviation and CAPA closure time and number of records created during a given period. The intent of this metrics is to assess the ability to close records on time and the volume received.
Unfortunately, very little emphasis is given to defining and implementing validation metrics that assess quality and productivity.
Validation metrics are often ignored because companies struggle to understand how to define meaningful validation metrics. This is often impacted by the lack of understanding of the importance of validation metrics and the inability to provide adequate definition.
Another issue impacting validation metrics is that companies normally do not track adequately validation activities during the lifecycle.
QUALITY METRICS IMPACT THAT VALIDATION
The FDA quality metrics initiative does have an indirect impact on validation. Meaningful validation metrics are a key indicator of the quality of the program and systems during their lifecycle.
Validation metrics are an early indicator of potential problems with processes and equipment that can have a negative impact on the product.
Validation metrics can detect very early issues that can be corrected before having a negative impact on the product and the FDA quality metrics.
Validation metrics enable the ability to assess and monitor the health and maturity of the validation program. Validation metrics such as number of repeat activities, invalid runs, execution errors, and others are key indicators of issues that impact the quality of the program.
Understanding of the health and quality of the validation program is a mechanism that enables the identification of corrective actions and continuous improvements initiatives.
The definition and implementation of meaningful validation metrics enable alignment with the FDA quality metrics initiative, understanding of the quality of the program and system during their lifecycle.
QUALITY METRICS FOR VALIDATION
Validation metrics can be categorized in the following categories:
- Quality Metrics
- Productivity Metrics
Quality validation metrics are those metrics that provide an assessment and measurement of the quality of the validation program. The quality of the validation program can be defined as the ability to execute the process with the highest level of quality. The quality of the validation program also includes the ability to meet the intent of the process which is to ensure that systems used in GxP operations performed as intended and according to user requirements.
Quality validation metrics should also include data that detect potential problems with processes and equipment.
The definition of quality metrics for validation is critical for the collection of critical that assess the validation program including processes and equipment.
Companies continue to struggle with the definition of quality metrics for validation. This is related to the poor understanding of key quality indicators for validation during the lifecycle of equipment and processes.
Companies should consider implementing the following quality metrics:
- Protocol deviations per month and year
- Protocol deviations per protocol
- Protocol deviation per person month/year
- Protocol deviation per system month/year
- Number of repeat activities per month/year
- Number of invalid runs month/year
The definition of these metrics is critical to ensure that adequate and accurate data is collected during the validation lifecycle.
Quality metrics for validation are an excellent tool to measure, assess and understand the process maturity, quality and ability of the process to meet its intent of ensuring that GxP systems are fit for intended use.
PRODUCTIVITY METRICS FOR VALIDATION
Productivity metrics for validation are often misunderstood and ignored in the industry. This is due to the lack of understanding of the key indicators that can be measured to assess the productivity.
Productivity metrics are intended to measure and assess the output and efficiency of the process. Productivity metrics for validation need to include the volume of deliverables and the cycle time related to the process. Some of the quality metrics also provide information about the efficiency of the process such as the number of document review cycles from reviewers.
The definition of productivity metrics should consider the following information:
- Number of protocols per month
- Number of protocols per person
- Cycle time
- Number of document review cycles
Validation productivity metrics are an excellent tool to identify bottlenecks and inefficiencies in the process. The productivity metrics can be used reduce cycle time and the cost of the validation program. This can be accomplished by defining target productivity metrics goals that are based on reducing or eliminating the inefficiencies found in the validation process.
IMPLEMENTING VALIDATION METRICS
The implementation of validation metrics requires definition of each individual metric that will be measured in the program.
Validation metric definition is a critical step to enable the implementation of meaningful metrics that provide accurate data as an indicator of the maturity and health of the validation program
Validation metric definition need to ensure that each individual value is clearly defined and understood by the organization.
Validation metrics that are not adequately defined create the risk of generating inadequate data that is not clear and accurate
Validation metrics definitions should provide a complete statement of what is going to be measured to clearly communicate to the audience the expectations. The following are examples of validation metrics definitions that can be considered during implementation:
- Number of deviations per protocol: The ratio of validation deviations found in executed protocols
- Number of repeat validation activities per protocol: Number of re execution needed per protocol due to issues during validation
- Total number of protocols per month: Number of protocols created, reviewed and approved during a given month
- Total number of protocols per person: Number of protocols created, reviewed and approved during a given month
Implementing validation metrics requires tools to collect and calculate the data based on the metric definitions.
Implementing validation metrics require an adequate implementation plan that provides a roadmap for the implementation. In addition to the implementation plan companies should also consider creating an adequate change management and communication plan. The change management and communication plan integrated with the implementation plan.
BENEFITS FROM IMPLEMENTING VALIDATION METRICS
The implementation of adequate validation metrics provides a significant amount of benefits to the industry and these include the following:
- Ability to measure and assess the quality of the program
- Ability to measure and assess productivity
- Ability to measure and assess the efficiency of the program
- Ability to define metrics targets that improve the quality of the program
- Ability to define metrics targets that improve the efficiency of the program and reduce cost
The implementation of validation metrics enables companies to get a better overall understanding of the validation program. Validation metrics also enable the ability to drive continuous improvements initiatives driven by the metrics and identified areas of opportunity.
Validation metrics are an excellent tool to measure critical aspects of the validation program. Validation metrics provide critical information during the lifecycle of GxP systems and processes.
To ensure an effective implementation of validation metrics it is critical to have an adequate definition for each metric.
The implementation of validation metrics requires companies to use electronic tools such as Excel or Validation Lifecycle Management Systems to collect the data needed to generate the metrics.
The implementation of validation metrics provides companies a significant amount of benefits and enable companies to have a good understanding of their validation program.
Disclaimer: The views of this articled expressed in this paper are those of the author alone and do no represent the views of Emergent.