A Comparison of Acceptance Sampling Plans and the Kappa Statistic in Attribute Test Method Validation


Current, popular statistical measures for the validation of inspection methods fall short in three ways. Either: (a) the validation metric combines the distinctly different elements of Type I and Type II error into one metric; (b) the validation metric cannot be directly interpreted to determine the effectiveness of the inspection; and/or (c) the validation metric is only a point estimate, with no visible consideration of power or uncertainty.

This paper introduces a sampling plan approach to Attribute Test Method Validation (ATMV) and shows its performance to be superior to the widely used kappa statistic. Passing the sampling plan approach validates that the inspection system meets the required performance level and addresses the three concerns highlighted above.


A familiar binary classification test for many people is the airport security scanner. Imagine a security scanner that has a knob to control the sensitivity level of what will be detected. The objective is to detect dangerous metal items such as knives and guns without tripping the alarm for harmless items such as forgotten pocket change and belt buckles. There are two possible input conditions and two possible responses in this scenario. The two possible input conditions are the passenger either IS or IS NOT carrying a weapon. The two possible responses from the scanner are either METAL or NO METAL is detected.

This content is only available to IVT members.
Get help maintaining your knowledge . Read More!

If you are already a member and you do not have access to this article, upgrade your membership.
Need help? Read our FAQs.

Product Added Successfully

This product has been added to your account and you can access it from your dashboard. As a member, you are entitled to a total of 0 products.

Do you want access to more of our products? Upgrade your membership now!

Your Product count is over the limit

Do you want access to more of our products? Upgrade your membership now!

Product added to cart successfully.

You can continue shopping or proceed to checkout.