Understanding Sources of Variation in Validation Protocols

Feb 2016

This is a follow-up to the January Cleaning Memo where we discussed variations in cleaning processes. This month we’ll cover variations in the cleaning validation protocol itself (variation in the protocol as well as in the execution of the protocol). Reducing variability involves first understanding the sources of variation in a protocol, and then designing that protocol to reduce the effects of that variability by either controlling the extent of variation or by designing the protocol to effectively deal with an extreme (worst-case) condition.

We’ll deal with identifying sources of variation first. Here are possible sources of variation (not necessarily exhaustive) in cleaning validation protocols; the specific ones applicable to your individual situation will depend on your understanding of your cleaning process.

  • Operation of cleaning procedure in protocol runs
  • Operation of cleaning procedure with specified challenges in protocol runs
  • Swab sampling operator
  • Rinse sampling operator
  • Access to swab sampling sites
  • Selection of swab sampling sites (worst cases)
  • Order of sampling (chemical vs. microbiological)
  • Lost or compromised sample
  • Spiked level for recovery study
  • Analytical method detection limit and quantitation limit
  • Linear range for analytical method
  • Surface anomalies of equipment (such as rouging, weld discoloration)
  • Visually clean inspection process (viewers, distance, lighting, and angle)
  • Selection of worst cases for grouping (matrixing) approach
  • Setting quantitative limits for active and cleaning agent

While this list appears to be much shorter than last month’s list of sources of variation in cleaning procedures, we should not forget that all those sources of variation in the cleaning procedure may also appear when that cleaning procedure is executed in a protocol.

So, how are these sources of variation controlled in a cleaning validation protocol? There are generally three approaches. One is design of the protocol (not the cleaning procedure itself, which we covered last month) to reduce variation in execution. A second is adequate detail in the protocol in instructions and precautions for those executing the protocol. A third is training for actions to be taken in the protocol.

I’ll cover the issue of the design of the protocol first. For example, proper execution of the protocol is one of the reasons that I prefer shorter protocols. By shorter protocol, what I mean is a protocol that only includes detail about the steps that are important for the protocol executors to know (and follow). For example, listing the acceptance criteria is critical for a protocol. However, giving the detail of all the calculations that are done to arrive at an acceptance limit for the active is not necessary. It is adequate to have a reference to another document where those calculations are detailed. Another similar example deals with sampling locations. The sampling locations should be specified in the protocol, perhaps by a verbal description and/or a schematic or digital photo. However, it is not necessary to include the rationale for selection of those sampling location. A rationale document, providing the reasons for selecting each sampling site (such as difficult to clean location, location for preferential transfer) can be a document that is referred to in the protocol itself.

If an approver (or a regulatory reviewer) wants to find out details about calculations for limits or about selection of sampling locations, that person can ask for that separate report. There is no compelling reason to include it in the protocol itself. My concern about including all this supporting documentation in the protocol is that it is not important for the person executing the protocol. Yes, those personnel executing the protocol need to know what the limits in the analytical samples are (so they can determine whether acceptance criteria are met). They need to know where to take the swab samples. But it is not absolutely necessary that they know what was involved in the calculation of limits or in the rationales for swab site selection. One concern I have is that including all these “extraneous” materials may cause the people executing the protocol to miss something critical because they automatically skip over sections not relevant to what they have to do in the protocol. Let me emphasize that there is nothing fundamentally wrong with including lots of extra detail in the protocol itself; it certainly won’t get you into regulatory trouble. However, other things being equal it is better to only put in the protocol those things that are necessary and/or helpful for those people executing the protocol.

That said, let me hedge my bets somewhat. I do like to include a few short sentences on subjects like acceptance criteria and sampling locations that may be helpful for those reviewing the protocol. For example, when I list the limits in the protocol, I might include a few sentences that explain “These limits were based on typical carryover calculations, utilizing the more stringent of a dose-based and a health-based safe threshold value for the active”. Or, for the sampling locations, I might say something like “The locations to be swabbed were selected based on locations that were more likely to have higher levels of residues following cleaning based on equipment complexity, prior operator experience, and/or ….” The purpose here is to indicate to any reviewer that the limit values and the sampling locations were not just arbitrarily selected.

Now to my second way to reduce variation in the execution of the protocol, that of having adequate detail in the actions to be taken by those executing the protocol. An example of this is taking a final rinse sample of the process rinse in a CIP cleaning process. Now this is something that I have to be careful about explaining. Yes, in this situation I would prefer to have a separate written procedure for taking rinse samples. However, the protocol itself would have more specific information about where the sampling valve is located (perhaps described in words as well as a digital image). I would also like to have information in the protocol so that the sampler knows when to take that final rinse sample. After all, if a “final” rinse sample is missed, the protocol may become an invalided run. In addition, certain precautions should be repeated, even though those precautions are explicit in the sampling SOP (this is the “belt and suspenders” principle). So, a swab sampling procedure may have a caution about not spraying gloves with isopropanol when the analytical procedure is to be TOC. A rinse sampling SOP (procedure) may have a precaution about adequately cleaning the outside of the valve and then adequately flushing the valve before taking a sample. However, it is best to repeat those same precautions in the protocol itself. After all, not only do you want to make sure the cleaning process is done correctly, you also want the execution of the protocol to be done correctly.

The third way to reduce variation in the protocol is to train, train and then re-train. While for cleaning validation we sometimes focus on training for the cleaning procedure operators, it is equally important to train others involved in executing the protocol. This might include the samplers, the analysts (chemical and microbiological), those involved in visual evaluation, the person responsible for completing the protocol checklist for manual cleaning processes, and the validation/QA/technical service people who have some overall responsibility for the protocol.

Note that how you approach the issue of reducing variability in the protocol execution will depend on the specifics of your situation, as well as your experience with past protocols.

Take me to the memos

from the year: