I find myself again in a position where I have to change the planned Cleaning Memo in order to address a critical issue about validation. That issue involves the FDA process validation guidance, issued in draft form in November 2008 and issued as “Revision 1” in January 2011. Specifically, what do those documents say about the number of qualification runs (I will use the acronym PPQ, for “process performance qualification”, since that is the term used by the FDA). There are several possible questions, including what the documents say explicitly, what is implied by the FDA in the documents, and what we can reasonably infer from the documents.
The impetus for me writing this Cleaning Memo is an article published by Jim Agalloco (perhaps the premier worldwide expert on pharmaceutical process validation) in Pharmaceutical Technology in the February issue (Volume 35, Issue 2, pp. 68-76). The article is entitled “Risk-Based Thinking in Process Validation”, and is available online at https://pharmtech.findpharma.com/pharmtech/Article/Risk-Based-Thinking-in-ProcessValidation/ArticleStandard/Article/detail/707062.
One basic argument of the article is that statistics should not be used in order to determine the required number of PPQ runs in Stage 2 (the Qualification stage) of process validation. I agree; that is a valid statement. However, that statement is made in the context of an assertion that in the 2008 draft guidance the “FDA held that the ‘rule of three’ is no longer appropriate and implied that more batches must be evaluated to provide the statistical confidence that is a central focus of the entire document.” emphasis added.
Now the first part of that statement is certainly accurate. The FDA has abandoned the so-called “rule of three”. It is the second part that I have problems with. Based on a reasonable reading of the guidance (either the 2008 document that Jim Agalloco was discussing or the 2011 revision), can it be claimed that the FDA implies that more batches than three must be required as part of the PPQ, or that a statistical evaluation is necessary in order to determine the number of PPQ runs?
I don’t read as an implication by the FDA or see a reasonable inference by the reader about the requirement for statistical determination of the number of PPQ runs. To further look at this issue, let’s see what the FDA document (the 2008 draft) explicitly say regarding statistical evaluation in the PPQ section of the guidance. In Section 2.b is the statement that “we strongly recommend firms employ objective measures (e.g., statistical metrics), wherever feasible and meaningful to achieve adequate assurance”. That is an explicit statement about the use of statistical methods, but it is not explicit that statistical methods are recommended for determining the number of PPQ runs.
Furthermore, the reference to statistical methods here cites use of statistics where those methods are “meaningful”; clearly statistical methods are not meaningful for determining the number of PPQ runs. I think it is clear that use of statistical methods for determining the number of runs is not implied by this statement. Could one infer that it is a recommendation for statistical methods to determine the number of PPQ runs. I guess that is possible, but it certainly is a stretch without corroboration from other parts of the guidance.
A second reference to statistical methods in the guidance is in Section 2.c, where it is stated that “The number of samples should be adequate to provide sufficient statistical confidence of quality both within a batch and between batches.” The statistics mentioned here explicitly applies to the number of samples taken, and not to the number of batches recommended. It may be possible to infer that the FDA is actually recommending more than one PPQ run, since the statement referenced here refers to “quality … within batches”. A possible inference is that if it refers to “batches”, then it must be more than one. However, more than one may just be two, and this is certainly not a suggestion that the number of batches be set using statistical methods. Another possible interpretation is that comparisons “between batches” could between a PPQ and batches made as part of the design/development phase of validation.
In that same Section 2.c is a statement that the protocol should discuss “Criteria that provide for a rational conclusion of whether the process consistently produces quality products.” The guidance continues that the criteria should include a “ description of the statistical methods to be used in analyzing all collected data (e.g., statistical metrics defining both intra-batch and inter-batch variability)”. The arguments I would make related to “inter-batch variability” in this statement are essentially the same arguments made in my prior paragraph regarding “quality … within batches.”
These are the three references to statistical methods in the 2008 guidance, and those statements are essentially unchanged in the 2011 revision. Note that there are other references to statistical methods, particularly in relation to “continued process verification” or Stage 3 of validation (what I commonly call “validation maintenance”). However, those statements have no impact on the number of batches or PPQ runs in Stage 2 (or Qualification).
Furthermore, the FDA has made it clear in a “Q&A on CGMP” (updated November 2009) that they are not setting a minimum number of validation batches. The question posed (question #5 in the list of questions) is “Do CGMPs require three successful process validation batches before a new active pharmaceutical ingredient (API) or a finished drug product is released for distribution?”. The answer (in part) is:
“No. Neither the CGMP regulations nor FDA policy specifies a minimum number of batches to validate a manufacturing process. The current industry guidance on APIs (see ICH Q7A for APIs) also does not specify a specific number of batches for process validation.
“FDA recognizes that validating a manufacturing process, or a change to a process, cannot be reduced to so simplistic a formula as the completion of three successful full scale batches. The agency acknowledges that the idea of three validation batches has become prevalent, in part due to language in its own guidance documents. However, FDA is now clarifying current expectations on process validation. The 1987 Guideline of General Principles of Process Validation is currently being revised to address this issue. The emphasis for demonstrating validated processes is placed on the manufacturer’s process design and development studies in addition to its demonstration of reproducibility at scale, a goal that has always been expected.
“However, a minimum number of conformance (a.k.a. validation) batches necessary to validate the manufacturing processes is not specified. The manufacturer is expected to have a sound rationale for its choices in this regard. The agency encourages the use of science based approaches to process validation.”
One would think that if the FDA was actually thinking about more than three or a statistically justified number, they would have stated so in this document (as well as in the process validation guidance document).
I also searched the web for published articles or presentations that discussed the new draft guidance. I found only one that mentioned using statistics to determine the number of validation batches, but it was mentioned only as one line in a slide presentation. Interestingly, Jim Agalloco wrote an article in the May 2009 Pharmaceutical Technology entitled “FDA’s Draft Guidance for Process Validation: Can It Be Applied Universally?”. In that article he praises the draft guidance as “refreshingly simple” and as supportive of “good science”. However, the emphasis in that 2009 article is application of the FDA guidance to systems such as sterilization and cleaning. In that 2009 article there is no mention of the use of statistical methods to determine the number of validation runs, or of a minimum number of runs greater than three.
I believe a careful review of the FDA guidance (both the 2008 draft and the 2011 revision), as well as related FDA documents, will show that it is an overstatement to suggest that the FDA implies (in either guidance document) that statistical methods should be used to determine the number of PPQ runs, or that three is the bare-bones minimum. The emphasis of the FDA on understanding of manufacturing processes, and on the design/development stage of validation, should be consentient with the possibility that one validation batch (one PPQ run) may be adequate. Of course, the manufacturer has to provide a rationale based on risks, which in turn should be based on process understanding gained in Stage 1.
Unfortunately, my concern with Jim Agalloco’s recent article does not stop with his assertions that the FDA implies that more than three batches are required. After discussing the advantages of a risk-based approach, in Table I he then proposes a minimum number of PPQ runs (1, 3, 5, 7 or 9 batches) for various types of manufacturing processes based on the type of manufacturing situation. For example, seven batches are proposed for a “biological fermentation or cell-culture process similar to one previously validated by the firm”. Now I admit that he presents these with the intent to “foster a dialog between industry and regulators”. However, if the FDA has stated that it does not intend to set a minimum number of batches, but wants a rationale from the manufacturer, why even discuss the possible number of batches based on a formula if we realize that every company might be different depending on their level of process understanding? Isn’t one focus of risk-based approaches to get away from a “one size fits all” approach?
Now I will further admit that it might be easier for a manufacturer to be told to do three runs; that way it requires no additional thinking. However, I believe the new process validation guidance offers opportunities for manufacturers to be more efficient in their process validation activities if they properly implement the design/development phase.
So, where does this leave us? First, let me repeat that neither Jim Agalloco or I (and I assume the three people cited as influencing Jim Agalloco’s paper – Phil DeSantis, James Akers, and Russell Madsen, all prominent people in pharmaceutical validation science) do not believe that statistical methods should be used to determine the number of validation runs.
Where the differences arise is in the understanding of what is in the FDA guidance document, specifically whether the FDA implies that statistical methods should be used to determine the number of PPQ batches, or whether that is a reasonable inference from the FDA documents. I would maintain that this is not the understanding of the FDA as reflected in their process validation guidance documents and in the referenced Q&A on CGMP. I also disagree with proposing a minimum number of batches based on a process classification system, although I do realize that since Jim Agalloco believes that the FDA is arguing for a statistical basis (which he indicates is a sample size of 30 batches), then his argument for reduced testing might be a reasonable step to try to reduce the burden on the industry. However, the burden on the industry is not to have a statistical basis for the number of runs. The burden on the industry is to select the number of PPQ runs necessary, as part of the overall validation effort, to (in the words of the FDA) “provide a high degree of assurance in its manufacturing process to justify commercial distribution of the product.”
Let me also state that I have exchanged emails with Jim Agalloco regarding this, and while we both agree that statistical methods are not the way to select the number of PPQ runs, we disagree on the implications or inferences possible with the FDA guidance.
Finally, some of you might ask why I am writing this when my expertise is cleaning validation, not process validation. The reason is that cleaning is a type of manufacturing process. While cleaning validation is not explicitly covered by the FDA guidance, I believe the principles expressed in that document should be considered as they are applicable to manufacturing equipment cleaning processes.
I would encourage interested readers to carefully read Jim Agalloco’s article, the FDA 2008 draft guidance and 2011 revision, as well this Cleaning Memo.