Courtesy of Getty Images
An effective audit leads to improvements in a company’s Quality System as well as its basic management practices. John Sawyer, in an article published in Medical Device and Diagnostic Industry magazine (May 1996) said:
“Of all the continuous improvement tools available, conducting quality audits is by far the best way for a company to ensure that its quality system is adequate and effective.1”
The benefits of internal auditing include:
- Reviewing department procedures to assess whether internal department controls are adequate
- Assessing system controls between departments
- Completing an independent assessment of the effectiveness and efficiency of a department
- Determining compliance with regulations (Food and Drug Administration (FDA), International Organization for Standardization (ISO), or others)
By conducting internal audits, companies take a proactive approach to managing potential compliance problems rather than being reactive to problems identified either by regulatory authorities or customers. Oftentimes, third-party audits focus on objectionable items rather than looking at potential system issues. This narrow view can lead the company into a false sense of security and cause them to fail to identify serious compliance problems.
Choosing the Team
It is important to have a sufficient number of well-trained auditors, since auditors cannot appraise their own areas. It is also beneficial to expose each area to different auditing styles. The total number of trained auditors required depends upon the size of the company; the larger the company, the greater the number of auditors that are required. Based on the author’s experience, a reasonable assumption is to have the number of trained auditors at approximately 5-6% of the total number of company employees. Therefore, if the company has 100 employees, five or six trained individuals are recommended.
Request a volunteer from one of the more difficult areas to be audited to serve as an assistant to the audit team. This serves several purposes. One purpose is to train the individual in the advantages of conducting the internal audit and in the audit process itself. Another purpose for using the individual is to gain a better understanding of the department’s organization, function, and general operations. To promulgate the company’s audit policies, it is helpful to allow this volunteer to train others in the company’s audit philosophy, explain the reasons audits are necessary, and to describe the useful purposes they serve.
The audit team has the responsibility for notifying the department to be audited, preparing an audit plan and associated checklist, conducting the audit, preparing the audit report, and performing followup on corrective actions developed by the audited department.
Beneficial Qualities for Auditors
The attributes of the individual audit team members should include the following:
- Professional attitude
- Open, above board
Among auditor qualifications that are most important are the following:
- Interrelationship skills; a good “people-person”
- Communication skills, both oral and in written form
- Investigational skills, including the ability to gather accurate facts and objective evidence
- Attention to detail
- Professional approach
- Flexible attitude
- Safety consciousness
The audit program should be described in the Quality Manual as well as in a policy document or Standard Operating Procedure (SOP). The audit program ensures that quality audits of all functional areas are performed at least annually.
Functional areas include:
- Quality Assurance
- Quality Control
- Sales and Marketing
- Customer Service
- Product Development
- Service, if applicable
The internal audit can be conducted during a single interval once each calendar year or a rotating schedule can be developed in order to spread out the auditing function throughout the calendar year. An example of the rotating type of schedule can be found in Figure 1.
|Function Area||Mgmt||QA||QC||S & Ma||Customer Service||PDb||Mfgc||Matd||Shipping||Receiving||Service||Follow-upe|
|No. of Managers||2||2||2||2||2||2||2||2||2||2||2|
|No. of Days||2||3||3||1||2||3||3||3||2||2||2|
aS&M: Sales and Marketing, bPD: Product Development, cMfg: Manufacturing, dMat: Material, eComplete any follow-up or recheck of any area
Purposes of internal audits may be preventive, detective, or corrective. Preventive audits are useful in the discovery of undesirable conditions before they result in problems. Detective audits are designed to investigate problems that have occurred either internally (adverse trends in quality data) or as a result of customer complaints. Corrective audits are necessary to ensure that actions are taken to reverse noncompliance or improve compliance. A combination of these three forms of audits can be a powerful tool to ensure that quality goals are consistently met.
Prior to Initiating the Audit
Before planning begins, it is essential to ensure that the company’s senior management supports the quality program and the regulatory requirements to perform the internal audit. Without senior management support, the internal audit cannot be effective and successful.
Prepare for the audit by reviewing all applicable regulations and guidance documents. Review key applicable company SOPs. Prepare audit checklists for the areas to be audited. Use published checklists, or create a custom checklist.
In preparation for the audit, time should be spent reviewing recent regulatory activities to determine whether there are new or continued areas of special concern. The FDA websites have links to recent Warning Letters and Recalls. It is helpful to review several of these recent Warning Letters to determine what weaknesses have been identified.
A Warning Letter issued June 28, 2004 to GF Health Products, Inc. stated:2
- Failure of management with executive responsibility to review the suitability and effectiveness of the quality system at defined intervals and with sufficient frequency according to established procedures to ensure that the quality system satisfies the established quality policy and objectives and the intent of the Quality System Regulation, as required by 21 CFR (Code of Federal Regulations) 820.20(c). For example, the only documentation showing management review of your Quality Systems were Certificates of Audit dated 8/21/02 and 2/21/03. Your procedure, “Management Responsibility,” (QPM-B1.B) states that “Management reviews shall occur on [redacted]. Additionally, your procedure, “Management Review of Quality Systems,” (QSP-B 101-QA.A) requires that “Management Review shall be held, at [redacted]. Similarly, your “Internal Audits” procedure, (QSP-B201-QA.A) states that internal audits are to be performed [redacted] in accordance with the Internal Audit Matrix. There is no evidence that your firm is conducting management reviews or internal audits according to the above procedures.
- Failure to appoint and document appointment of a management representative, as required by 21 CFR 820.20(b)(3). Procedure QPM-B1.B, Management Responsibility, requires that your firm appoint a Management Representative who [redacted]. Your firm could not provide any documentation showing the appointment of the Management Representative, nor evidence that management reviews or internal audits of the Quality System have been performed according to your procedures.
A Warning Letter issued to Positron Corporation April 26, 2004 stated3:
- Failure of the management with executive responsibility to provide adequate resources, including the assignment of trained personnel, for management, performance of work, and assessment activities, including internal audits [21 CFR 820.20(b)(2)]. For example: a) You failed to hire new employees or reassign existing employees to ensure that management reviews and internal audits are being conducted at defined intervals [FDA-483 Items 1 - 3]; b) You hired a new Quality System Manager, but did not provide this employee with quality system training [FDA-483 item 7]; and c) You terminated an employee who handled your firm’s customer complaint handling but failed to hire new employees or reassign existing employees to maintain your firm’s complaint files. An electronic customer service database was created to document customer complaints, but it was never implemented [FDA-483 Item 6].
- Failure to conduct and review internal quality audits according to established procedures to assure that your firm’s quality system is in compliance with the established quality system requirements [21 CFR 820.223 [FDA-483 Item 1, 2], a similar deviation from the previous inspection in 11/2001. For example: a) Only two out of ten quality system areas were audited in 2003. For example, radiation safety, CAPA, inspection and test status, control of non-conforming product, device history records, product identification and traceability, and document and data control were not audited; and b) Some audit reports were not reviewed in 2002 and 2003. For example, document and data control and human resource report in 2002 and engineering report in 2003.
- Failure to conduct management reviews at defined intervals according to established procedures [21 CFR 820.221 [FDA-483 Items 3 and 6], a similar deviation from the previous inspection in 11/2001. For example: a) Only one out of [redacted] scheduled management reviews was conducted in 2003, and no review was conducted in 2002; and b) An electronic customer service database was not implemented to document and evaluate customer complaints as required by your firm’s Complaint Handling Procedure (P.O.S. 4.14.03). This deviation was not detected by either management reviews or internal audits.
Preparing for the Audit
To ensure the efficiency and the effectiveness of the internal audit, an Audit Plan is recommended. (See example Audit Plan in Figure 2 following this article.)
The audit plan should include the following elements:
- Purpose of the audit
- Areas to be included in the audit
- Number of auditors required
- Names of auditors
- Proposed dates and duration for the audit
- Proposed date for the opening meeting
- Planned date and time for the wrap-up meeting
- Planned date for delivery of the audit report
- Standards, regulations, and guidance documents to be used
- Example checklists
During the development of the audit plan a meeting with the auditors is held to discuss the objectives and the details of the audit being planned. Auditors must review the reference documents that will be used to audit against as well as any applicable SOPs for the areas being audited. The team must identify the appropriate and meaningful elements to be included in the audit. Once the areas of focus have been identified, area-specific checklists can be prepared to guide the auditor through the items to be checked. The checklists that are created should be used as a guide only, and the audit should not be limited to only those items shown on the checklist. It is recommended that the auditor share the checklist with the department or functional area being audited to help them prepare for the audit.
Some potential obstacles include:
- Failure to Have Authority. At times, it may not be clear that the auditors have the authority to conduct a comprehensive audit of each functional area. Quality Assurance should have the authority to identify and provide recommendations for Corrective and Preventive Action (CAPA). Since the auditors have experience with the types of corrective and preventive actions that have been previously demonstrated as effective, any recommendations provided should be strongly considered.
- Lack of Independence. It is important that auditors maintain independence and that the audit function is not compromised by senior management pressure. The auditors must be able to accurately report their findings regardless of the seriousness of the items found.
- Perception of Not Being Part of the Team. Since auditors often identify weaknesses in the department being audited, some may perceive the auditors as not being team players.
An opening meeting is scheduled with the auditees. It is important that the individuals being audited understand the advantages provided by the internal audit and that they understand the importance of providing complete and honest answers to the questions asked by the auditor.
In the opening meeting, discuss the following:
- Purpose of the audit
- Areas and items to be audited
- Proposed schedule and duration
- Individuals from the audited area whose availability will be required
Performing the Audit
It is important for the auditor to plan the required reviews in the time allowed. It is important for the auditor to “manage” the audit in a manner that will allow him or her to complete all aspects of the audit. If the auditee has a tendency to provide additional information and details beyond what is requested, it is the responsibility of the auditor to make sure the auditee limits his or her response to only the question(s) being asked. Superfluous information is not helpful. The auditor must manage the time devoted to the audit to ensure all aspects are completed according to the scheduled time allotted.
One difficult aspect of auditing is knowing when to call for interviews and when to spend time reviewing materials. How this is handled becomes a personal preference during auditing. Personally, I have found it useful to request documents, spend time reviewing them, jot down questions, tag areas that are unclear or create questions, and then request that the Subject Matter Expert (SME) review the questions compiled. This style of auditing can (and should) create an efficient method of auditing. There are times when the materials being reviewed are not well organized and do not lend themselves to a desktop review without the explanation of the SME. In these cases it is necessary to have the SME meet with the auditors to explain the document package.
At the first sign of not having the appropriate person available to address the audit questions, it is the auditor’s responsibility to stop the questioning and request the appropriate person who is knowledgeable in the area being audited.
Some useful suggestions to find potential weaknesses include:
- Start with a “walk-about.” Take this opportunity to review the general flow of personnel and material. Learn how material is stored and separated, and what individuals are doing; determine whether documents are located in the areas where work is being done, and how material and product is labeled; establish the general levels of organization and cleanliness, etc.
- During the “walk-about,” jot down the lot numbers of materials in use, in progress, in storage locations, etc. Use these lot numbers to request manufacturing and test records. Areas of particular interest include, quarantine, returned goods, and Material Review Board (MRB) items, especially if the items have been in those locations for any period of time.
- Review complaints, nonconformance and deviation reports, and materials in quarantine. Look for the timeliness of identifying problems, investigating the potential causes, and the completion of corrective actions.
- Review items that have been assigned to MRB and the decisions that have been made on the use of the material.
- Review the design history files or product reports for recently developed products or changes to existing products.
- Review the frequency at which management review or product review meetings are held and the agenda for those meetings.
- Review environmental controls and records for environmental monitoring. These records may reveal problems with cleaning, personnel training, etc.
- Review validation master plans and the dates when the validations were completed. Check that the validations for test methods used in the validations and the equipment required for the process were completed prior to the process validation.
Reviewing these areas first will identify many of the significant problems, if they do exist.
Executing the Audit
Areas cannot be successfully audited when there is inadequate documentation of the operations per formed in the areas being audited. There is no way to determine whether a process is operating in a state of control if there are no procedures describing the steps to be performed and no records to document what was actually done. One FDAer (Gene Murano, CBER) once said at a regulatory conference: “In God we trust, everyone else brings data.”
As the auditor conducts the audit, the auditor should keep a list of the individuals (name, department, title) that he or she talks with and the date of the interview. The auditor should also maintain a running list of the documents that he or she reviews as part of the audit. The complete title of the document should be listed along with the document number, effective date, and edition (or revision).
If deficiencies are identified, objective evidence must be retained. The objective evidence should be well-marked in order to provide traceability between the observation and the documentation. Complete copies should be made and maintained until the close of the audit.
When reviewing records, it is essential that the auditor is aware of the version of the document that was current at the time the record was completed. It is a common mistake by auditors to fail to make this important verification check at the time of the audit. It is important to note that the documents provided on the day of the audit may not have been in effect at the time the records being reviewed were completed. As part of the objective evidence gathering, it may be necessary to request and retain copies of older versions of the supporting documents.
When an observation is made that suggests noncompliance, discuss the observation with the SME to be sure that there is not a misunderstanding with the record or supporting procedure. It is always best to clear-up any possible mix-up before writing or presenting an observation to the audit team leader.
At the end of each day, it is recommended that the auditor meet briefly with the department head of the area being audited to discuss the findings made that day. It is important that the head of the department not be surprised by hearing of observations from a third party.
Following the brief meeting with the department head, the audit team should also meet to discuss the collective observations made for that day. This team meeting allows for the review of any potential systemic problems and gives an opportunity for auditors to discuss their opinions regarding the observations. In cases where there is a difference of opinion as to whether a finding is or is not an observation, it is the responsibility of the audit team leader to make the determination, to request a review of regulations, or to request additional information prior to making the final determination.
It is helpful to have an established format to report audit observations. The format may be similar to the example provided in this article. (See Figure 3 following this article.) It is best that the observation forms be completed at the end of each day of auditing. Depending on the need for verification of audit findings, this may or may not be possible. Effort should be made to expedite the completion of the audit report upon completion of the audit.
Once written, the observation form should be reviewed to ensure that the observation is clearly and accurately stated to avoid any misunderstanding. Details of the observation should be listed using clear and concise language. Using the present tense and active voice is recommended, rather than using past tense and passive voice. Words such as “always,” “never,” “all,” “every,” etc., should be avoided. Rather than referring to individuals by name, titles or department names should be used.
Include a description of the records reviewed, as applicable, that resulted in the observation. Specific identifiers should be used whenever possible, such as document numbers, editions, effective dates, page number, section or paragraph number, etc. Be as specific as possible. For example, when referring to an error or inconsistency in a standard operating procedure, state: [An inconsistency was located in SOP QA-012, edition 091205, effective 092005, entitled “Preparation of a Manufacturing Instruction,” on page 3 of 8, section 8.1.1. The document states that section 9.0 of the Manufacturing Instruction is titled “Reference Documents” when the correct title of this section according to the example template attached to the SOP is actually listed as “Equipment.” The correct section title was confirmed with the Document Control personnel as “Equipment.”]
Distinguishing between Minor and Major Observations
Most commonly, observations are separated into two levels of significance: minor and major. Minor observations are those which, by themselves, are not likely to be considered regulatory violations; however, collectively may result in a “483 observation” (or regulatory violation as listed on FDA Form 483). For example: if there is only one instance of illegible record keeping, this would be considered a minor observation. If there are multiple examples where data cannot be easily interpreted, this minor observation would become major.
A major observation is an observation that by itself, is considered a regulatory violation that would likely be a 483 observation. An example would be a Medical Device Report (MDR) that was not submitted to the FDA.
Prior to the wrap-up meeting, a pre wrap-up meeting is held with the auditors to discuss the collective observations, significance of the observations, and what recommendations can be provided for the documented observations. It may be necessary to combine minor observations made in a single quality system area that will result in a major system observation, such as Management Responsibility or CAPA.
All individuals who participated in the audit should be in attendance at the wrap-up meeting. Prior to beginning the presentation of the observations, the positive aspects of the audit should be presented.
Positive comments may include:
- Cooperation of auditees
- Flexibility of area managers
- Efficiency of locating requested records
- Orderliness and cleanliness of areas
- Improvements over previous audits
During the presentation of the observations, it is important that the participants understand the observation, agree with its accuracy, and are clear as to how and why the area was found out-of-compliance with the regulation. Any discrepancy between the observation and additional information provided during the wrap-up meeting must be resolved prior to the issuance of the final report.
Final Audit Report
Following resolution of any follow-up items from the wrap-up meeting, the final audit report is prepared. The format of the audit report may vary widely depending on the purpose of the audit, the depth of the audit, etc. One suggested outline is the following:
- Cover page
- Table of contents
- Executive summary including an abbreviated list of observations with a notation as to their significance
- Background (purpose and scope of audit, etc.)
- Section by section analysis of the quality system elements describing what was reviewed and what was found
It is very important that the audit report not include any items not discussed during the wrap-up meeting or not discussed with the auditees. In the event there was an observation inadvertently left out of discussion with the auditees during the audit, a follow-up call should be made to the auditees and to the area supervisor or area manager prior to including the observation in the audit report. The management personnel of the audited area should be fully aware of all observations prior to the issuance of the report.
Corrective Action Follow-up
As part of the audit process, a request should be formally made to the audited department for a corrective action plan. The corrective action plan should include actions for each individual observation as well as a quality system correction, where applicable. The plan must focus on the “root cause” of the observation. It is important to identify the “right” problem and correct the problem with the “right” solution. It is not uncommon to find companies working to fix the “wrong” problem.
The corrective action plan must be provided in a reasonable timeframe following receipt of the audit report. A reasonable timeframe for typical audits is two weeks. In the event there are a large number of observations, an extension may be required to allow the audited department sufficient time to prepare the action plan. In all cases, each action must include a reasonable timeframe for completion. The timeframe will vary with the extent of the action required. It is always desirable to have actions completed within a six-month timeframe with more simple actions being completed within a one-month timeframe. In all cases it is important to be able to demonstrate progress on the corrective actions.
One individual from the audit team should be assigned to track the completion of the corrective action tasks. The commitment to complete the actions should not be taken lightly. The designated monitor must notify his or her supervisor immediately if or when corrective actions are not completed by the specified target date. If the target date for completion is missed by the audited department, the area supervisor must provide a reasonable explanation for the delay. A history of delays in the completion of the action items is a clear demonstration of the lack of commitment to support regulatory compliance and this should not be tolerated by senior management. Conformance to regulatory requirements should be a condition for continued employment. Individuals who fail to comply create a liability for the company and disciplinary action or termination should be considered.
It is not only important for the corrective actions to be the “right” solutions and completed on time. It is also important that the corrective actions are verified as being effective. Once the corrective action plan is received, the designated audit team member should be assigned to the development of an effectiveness check for each of the corrective actions. As the action items are completed, the effectiveness checks should be scheduled with the audited department. The effectiveness checks should be scheduled such that sustainable compliance can be demonstrated. Conducting the effectiveness check too early will not be beneficial to the audited department or to the quality program. Allow sufficient time to develop evidence (i.e., records) demonstrating that the improvements correct the problem.
Feedback regarding the Audit and the Auditors
The last step in the audit process is to obtain feedback from those audited in order for improvements to the auditing process to be realized. Feedback can be requested either informally or formally. Informal feedback can be obtained during personal discussions with the audited department. These discussions should be maintained as confidential to protect those interviewed and the input should be used for constructive purposes. Formal documented surveys are another mechanism that may be useful to solicit input from those audited. Feedback should be requested on the auditors, the auditing style, the general organization of the audit, the conduct of the audit, the audit schedule, the disruption to routine department activities, etc.
Just as products have a lifecycle, audits do as well. There is a loop that must be closed for each audit, which includes the careful planning of the audit in order to assess the “right” areas, the accurate identification of potential problems, the assignment of the “right” corrective actions, verification of the completed corrective actions, and follow-up on those corrected areas during the next scheduled audit. Continuous improvement should be the goal following any audit.
- Sawyer, J. “Conducting Internal Audits: Taking the Company’s Pulse”, Medical Device & Diagnostic Industry, May 1996.
- FDA Warning Letters and Responses (accessed: November 2005)
- FDA Warning Letters and Responses: (accessed: November 2005).
- Code of Federal Regulations, 21 CFR 820
- “Establish an Internal Audit Program,” isostore.com
- 3. Green, C. “How to Manage an FDA Inspection for Medical Devices,” Journal of Validation Technology, Special Report.
- 4. Guidelines for Auditing Quality Systems-Part 1: Auditing, ISO 10011:1990.
- 5. Guidelines for Auditing Quality Systems-Part 2: Qualification Criteria for Quality Systems, ISO 10011-2: 1991.
- 6. “Internal Audit and Control FAQs,” Asistl.com.
- 7. Wells, T.R. “Is the U.S. Meeting the Global Harmonization Guideline for Auditing Quality Systems?” Medical Device & Diagnostic Industry, October 2000.
Figure 2: Example Audit Plan
|Title of Audit:||Provide the name of the audit. For example, "Annual Compliance Audit." Be consistent with the terminology used in the Internal Audit Standard Operating Procedure.|
|Proposed Date(s) of Audit:||Include the full range of dates when the audit will be conducted or is planned.|
|Department (or Area) to be Audited:||List the department(s) that will the subject of the audit.|
|Scope of Audit:||Indicate the scope of the audit. List an exclusions, such as specific product lines, production lines, etc.|
|Applicable Regulations or Standards:||List the regulations that will be applied, such as 21 CFR Parts 200, 600, 800, etc.|
|Audit Team Members:||List all members of the audit team by full name, title, and department or affiliation.|
Figure 3: Example Audit Observation Form
|Title of Audit:||Provide the name of the audit, for example: "Annual Compliance Audit." Be consistent with the terminology used in the Internal Audit Standard Operating Procedure.|
|Date(s) of Audit:||Include the full range of dates when the audit was conducted.|
|Audit Team Member:||List the full name of the auditor as the author of the observation.|
|Date of Observation:||State the date that the observation was made. If the observation was discovered on one day and then verified at a later date, record the first date of the observation.|
|Observation Number:||For tracking purposes, each auditor should number their observations. A tracking number should include the auditor's initials or auditor number along with a sequential number for each observation.|
|Observation: Record the observation providing all applicable details.|
|Applicable Regulation: Include the section of the Code of Federal Regulations (CFR) that is applicable to the observation.|
|Assessment of Significance: Provide an initial assessment of the level of significance for the observation.|
|ARecommendatiosn for Corrective/Preventive Action: List some applicable recommendations. Ensure that the Corrective and Preventive Action (CAPA) is appropiate for the "root cause." If the root cause cannot be easily determined, indicate that an investigation is required to determine root cause and corrective and preventive actions.|