A pass/fail test method is an analytical test method that only tells me whether I am below a certain value. For example, I may have a HPLC method that strictly tells me that I am at or above a value of X ppm in my sample, or I am below a value of X ppm. In the context of cleaning validation, if the limit in my analytical sample (a swab extract, for example) is X ppm, that pass/fail test method should allow me to say whether I am meeting my acceptance criterion, assuming that my sampling recovery is 100% or that I have dealt with a lower recovery by other means (see below).
This pass/fail method may be a detection limit test procedure. That is, I am strictly measuring whether I am below the detection limit. If the detection limit is at or above the acceptance limit in the protocol, demonstrating that I am below that detection limit should be evidence that I am meeting my protocol acceptance criteria. Another type of pass/fail test method is one where I am comparing the measured response of the test sample to the measured response of a standard representing the 100% concentration (ppm) of my acceptance limit. In other words, if the acceptance limit is X ppm of my active, I prepare a standard at X ppm of that active. I run that standard and my test sample in my HPLC procedure, and compare the responses (area under the curve or peak height). If the response of my test sample is less than the response of my standard, then I have demonstrated that the test sample is meeting my acceptance criterion. Note that generally that standard is carried along for every set of test samples that are analyzed. I may also prefer to run a standard before and after my test samples (that is, I bracket my test samples with injections of my standard).
This is generally not what most companies do for analytical methods for cleaning validation purposes. Most companies prefer to have a quantitative method that is validated to be accurate and precise over a certain range. One reason for doing so is to demonstrate the robustness of the cleaning method. For example, if my limit is X ppm, I have much more confidence in the robustness of my cleaning process if I have data that shows samples are consistently in the range of 0.1X-0.3X ppm as compared to just be able to say that I am only below X ppm (as in a pass/fail method). A second reason for having a method validated over a certain range is if I am using a “stratified sampling” approach to setting limits (see the Cleaning Memos of March, April and May 2010). However, the point is that at least from a scientific point of view, a pass/fail method may be acceptable. One situation where a pass/fail method might be used is in a cleaning verification mode for clinical trial material manufacturing, where spending time and effort for a method validated over a range may not be required.
Now we’ll cover one of the main concerns in the use of a pass/fail test method, the issue of dealing with sampling recovery. That is, my pass/fail point is X ppm; but suppose my recovery might be as low as 50%. If my swab test sample shows that I am below X ppm, how can I say I am meeting my acceptance limit if my sampling recovery is only 50%, or for that matter any recovery percentage value below 100%?
What can I do to avoid this recovery problem with pass/fail test method? The simplest way is to just set the pass/fail criterion in the test method at something below the calculated acceptance limit in the analytical sample. For example, if my calculated swab limit is X ppm, I can assume a worst case sampling recovery of 50% and establish my pass/fail value in my test procedure as 0.5X ppm. Note that this doesn’t work if my pass/fail value is solely a true detection limit value; in that situation I cannot establish that I am meeting my acceptance criterion if my sampling recovery is less than 100%.
But, you ask, what if my sampling recovery is actually less than 50%? Don’t I have a problem there? The answer is that I don’t just assume a 50% (or higher) recovery. I do something to demonstrate that my recovery is at least 50%. How is that done? Very simply. I first establish my pass/fail method based on a value of 0.5X ppm rather than X ppm. Then I perform a recovery study by spiking at a level representative of the value of the X ppm limit. I then perform my analysis on that test sample with my pass/fail test procedure. As long as my result shows a value at or above 0.5X, then I have clearly demonstrated that my sampling recovery is at least 50%. In this example, if I spiked at an equivalent of X ppm, and the measured value in my recovery study was below 0.5X ppm, I would have demonstrated that my sampling recovery in this case was less than 50%. Therefore, in the latter case, using that combination of a pass/fail value of 0.5X ppm and the sampling method would be inadequate to demonstrate that I was meeting my acceptance criterion of X ppm.
In the example presented above, the pass/fail was established at 0.5X ppm. If I had a requirement that I must have a recovery of at least 50%, I could establish a pass/fail value of 0.5X ppm, or I could establish a higher pass/fail value. That is, I could still spike my coupon at X ppm, and then analyze with my pass/fail method at 0.7X ppm. If I showed a value of 0.7X ppm or greater, then I have established a recovery of at least 70%.
Therefore, the pass/fail method at 0.7X ppm could be used.
A second issue in the use of a pass/fail test method relates to demonstrating the robustness of my cleaning procedure. That robustness may be established by setting the pass/fail point at a value lower than the calculated limit. For example, that lower value may be 0.8X ppm or 0.6X ppm. Note that this would be in addition to dealing with the sampling recovery issue by using a lower value.
Note further that this establishment of an adequate recovery will generally only work in cases where the pass/fail limit is established by running a known standard and that an adequate measured response is obtained at a level below the ppm acceptance limit. In cases where the pass/fail point is a true detection limit, this method of establishing adequate recovery will not work.
One last discussion point is that a pass/fail method still requires analytical method validation. However, the method validation for a pass/fail method is generally less time consuming as compared to a method validated over a wide range.
The point of this Cleaning Memo is not to recommend the use of a pass/fail analytical procedure. Rather it is to present conditions to reliably and appropriately use such pass/fail methods in validation and/or verification protocols.