The investigation of out-of-specification (OOS) results is an important part of the work undertaken by the analytical laboratory. The OOS process is concerned with the examination of any result that falls outside established acceptance criteria. This either relates to acceptance criteria (‘specifications’) established in official compendia, such as pharmacopeia, or by an organization. The scope of the OOS also includes acceptance criteria established in drug applications and drug master files. There are other associated terms, such as ‘out of trend’, which are examined below.
Since 2006, regulatory agencies have produced guidance on conducting OOS investigations. In addition, many laboratories have established internal procedures so that OOS investigations are consistently undertaken. Indeed, the investigation of an OOS should be covered by a Standard Operation Procedure (SOP) and formally documented. The SOP should contain decision tress to ensure that, where possible, the conclusions reached are consistent.
Despite the guidance that is in place, many regulatory cite poor OOS investigations and these features high up on lists of inspectorate findings. Failure to conduct detailed OOS investigations or not producing OOS investigations of sufficient quality regularly features among the top five inspection findings from European Union regulatory agencies and also from the U.S. Food and Drug Administration. Notably these are renewed calls by agencies for the OOS investigation to be thorough, timely, unbiased, well documented and scientifically sound. Hence key challenges for many companies is with having a clear understanding of regulatory expectations on how to handle OOS investigations. Moreover, a lack of consistency around investigations and root-cause analysis processes will lead to error and expensive laboratory activities. To help, this paper takes a look at how OOS are conducted and presents different ways through which OOS investigations can be improved. While the OOS concept discussed is generally more applicable to analytical data than microbiological data, there are aspects in this paper that will be of interest to all laboratory disciplines working in a regulated GMP environment. The paper provides some best practice tips and short case study.
Importantly, at which ever stage the OOS is confirmed, a confirmed OOS result is an indication that the batch does not meet established specifications and therefore the batch needs to be rejected. Other information gathered during the investigation may call into question the disposition of other batches.
Differences in terminology
As well as the OOS, other related terms in common use in laboratories are:
Out of limits or out of level (OOL): This is a result that is above an established monitoring level, such as an action level as might be used to assess environmental monitoring.
Microbial data deviation (MDD): This is similar to the above OOL, but a term specific to microbiological test. It refers to any result that exceeds a microbial limit.
Out of expectation (OOE): This is an atypical, aberrant or anomalous result that lies within a series of results. The OOE result will meet specifications, but it falls outside the expected variability of the analytical procedure and may warrant examination.
An alternative descriptor is atypical, or aberrant, or anomalous results. These refer to similar things, relating to test results that are within specification, but which are also considered to be unexpected, questionable, irregular, deviant or abnormal. An example is a chromatogram that displays unexpected peaks.
Out of trend (OOT): An OOT is a time dependent result that falls outside data across a time period or a result detected as falling outside a statically derived limit ona process control chart. These may or may not be OOS
With reference to the microbiological issues, most OOS guidance is limited to chemistry-based laboratory testing of drugs. A reason for this is, as Sutton showed, due to the greater variability of microbiological data, which relies on less precise methods (in terms of metrology) and the need to take multiple samples and often, with replicate samples, to average results (1).
In this context OOS is applicable to:
- Batch release testing and testing of starting materials.
- In-process testing where generated data is used for batch calculations.
- In-process testing where samples appear in a dossier and on Certificates of Analysis.
- Stability studies on marketed batches of finished products.
- Stability studies relating to active pharmaceutical ingredients.
- Previous released batch used as reference sample in an OOS investigation showing OOS or suspect results.
- Batches intended for clinical trials.
Where did OOS come from?
The history of OOS has been well document (such as by Masden) (2). To summarize briefly, a generic drug manufacturing organization in the U.S. called Barr Laboratories had a history of repeated GMP deficiencies, and these included several instances for retesting and resampling of products. These led to the reprocessing of defective product without suitable justification (essentially testing into compliance). This led to dispute with the U.S. Food and Drug Administration (FDA), and Barr proceeded to sue the FDA for “ad hoc” drug regulation. The FDA applied to the court for a “stop ship” injunction, claiming that Barr products were adulterated. The case was heard by Alfred Wolin, Federal District Judge (New Jersey District). This became: United States of America v. Barr Labs, Inc. 812 F. Supp 458, 3/30/93 (3).
Fundamental to the ruling was the finding: “Barr’s practice of responding to an out-of-spec result by testing twice more and taking “best two out of three” is insupportable and unscientific and is not GMP.” From this Barr was forced to recall 12 products in commercial distribution and the FDA was required to develop suitable guidance for industry. The FDA produced "Guidance for Industry - Investigating Out of Specification Test Results for Pharmaceutical Production" (FDA, 1998) (4), which was reissued in 2006 (5), and it has not been updated since. Subsequently other regulatory agencies produced guidance or added requirements to GMPs. This includes the International Conference on Harmonization of Technical Requirements for Registration of Pharmaceuticals for Human Use (ICH), through ICH Topic Q 6 A “Specifications: Test Procedures and Acceptance Criteria for New Drug Substances and New Drug Products: Chemical Substances (6).”
Information pertaining to the importance of OOS results and specifications is contained with the Code of Federal Regulations. For example, CFR 211.22 states that the Quality Unit is responsible for approving or rejecting all components; a review production records (that no errors occurred); and for approving or rejecting all procedures or specifications. Moreover, CFR 211.160 General Requirements discusses the establishment (and change) of specifications, standards, sampling plans, and test procedures; CFR 211.160(b)(4) states what is necessary for the analyst; and CFR 211.165 outlines the review of specifications as part of lot release. To add to this CFR 211.165(f) specifies that products that fail to meet established standards and other relevant quality control criteria will be rejected. Even if a batch is rejected, an investigation is required to determine if the OOS is associated with other products or lots.