Validation with Total Error method

Validation

The different regulations concerning to the good practices (GLP, GMP, GCP, and others) as well as the normative or regulatory documents (ISO, ICH, EMEA, and FDA) suggest that all procedures have to comply with acceptance criteria. This request imposes, therefore, that these proceduresmust be validated. There are several documents defining the validation criteria to be tested, but they do not propose experimental approaches and limit themselves, most often, to the general concepts. It is why the members o the SFSTP have contributed to the elaboration of consensus validation guides to help the pharmaceutical industry to validate their analytical procedures.

This approach allows to considerably minimize the risk to accept a procedure that would not be sufficiently accurate or, to the opposite, to reject a procedure that would be capable.

The accuracy profile method, uses ISO terminology and offers an experimental strategy for the validation of analytical procedures, independent of the industrial sector, to optimally use experiments performed, to extract a maximum of information from the results and to minimize in routine the risks to re-analyze samples.

Objectives of an analytical procedure

In order to specify the objectives of the validation, it is necessary to go back to the nature itself of an analytical method. Is its objective to demonstrate that the response varies linearly as a function of the concentration, that the bias and the precision are less than x% or rather to quantify as accurately as possible each unknown quantity? These interrogations seem to be the questions of interest. The objective of a “good” analytical procedure is to be able to quantify as accurately as possible each of the unknown quantities that the laboratory will have to determine. In other words, what the analyst is seeking is that the difference between the measured value $x$ and the true value $\mu_T$, which will always remain unknown, is as low as possible or at least lower than an acceptable limit. This requirement can be expressed as follows:

    -\lambda < x - \mu_T < \lambda \Leftrightarrow  | x - \mu_T | < \lambda

with $\lambda$, the acceptance limit which can be different depending on the requirements of the analyst or the objective of the analytical procedure. Indeed, the acceptance limit can vary according to the intended use of the analytical method (e.g. 1%–2% for the analysis of a bulk pharmaceutical compounds, 5% for the determination of active ingredients in dosage forms, 15% in bioanalysis, etc.)

Important concepts are thus introduced, not only acceptance limits for the performance of an analytical method but also the responsibility that the analyst has to take in the decision of accepting the performance of the method with respect to its intended use.

On the other hand, every analytical method can be characterized by two types of error :

• a systematic error or true bias $\mu_M$
• a random error or true variance $\sigma_M$

An estimation of the method bias and variance can be obtained from the experiments carried out during method validation. The reliability of these estimates depends on the adequacy of the measurements performed on known samples, called validation standards (SV), the experimental design and the number of replicates during the validation phase. On the basis of these estimates for bias and variance, the acceptance limits for the performance of the method $\lambda$, it is possible to define the concept of good analytical method for a given field (e.g. biopharmaceutical analysis).

The example below shows the implementation of the acceptance criteria for different configuration of bias and precision.

Aiming to develop a procedure without bias and without error has a considerable cost. This target is unrealistic for an analyst who has generally only little time to systematically and meticulously optimize all the analytical parameters in the development phase even if the use of experimental design is recommended and well described in the literature.

To overcome this dilemma, the analyst will have to take minimal risks (or all at least compatible with the analytical objectives). To control this risk, the reasoning can be reversed and one can fix as starting assumption that only an acceptable maximum proportion of future measures will be outside the acceptance limits, e.g. 5% of the measurements or 20% of the measurements to the maximum outside the acceptance limits. This proportion represents, therefore, the maximum risk that the analyst is ready to take.

A procedure can be qualified as acceptable if it is very likely, i.e. with a guarantee, that the difference between every measurement $x of a sample and its true value $\mu_T$ is inside the acceptance limits predefined by the analyst. This concept can be described by the following expression:

    P(|x − \mu_T | < \lambda) \geq \beta

with $\beta$ the proportion of measurements inside the acceptance limits and $\lambda$ the acceptance limits fixed a priori by the analyst according to the objectives of the method. The expected proportion of measures falling outside the acceptance limits evaluates the risk of an analytical procedure.

Objective of the validation

The objective of validation is to give to the laboratories as well as to regulatory bodies guarantees that every single measure that will be later performed in routine analysis will be close enough to the unknown true value of the sample to be analyzed or at least that the difference will be lower than an acceptable limit taking into account the intended use of the method.

The goals of the validation are thus to minimize the consumer risk as well as the producer risk. Consequently, the objective of the validation cannot be simply limited to obtaining estimates of bias and variance but must be focused on the evaluation of the risk even if these estimators are needed to evaluate the risk. With respect to this objective, two basic notions mentioned above have to be explained: close enough, meaning, for example, that the realized measure in routine will be to less than $x \%$ of his true value unknown; guarantees, meaning that it is very likely that whatever the measure, it will be close enough to the true value.

In that respect, trueness, precision, linearity, ... are no more statistics allowing to quantify these guarantees. In fact, one expects from an analytical procedure to be able to quantify and not to be precise, even if the precision itself unquestionably increases the likelihood to be successful. In this perspective, it is necessary to differentiate the statistics which allow to make a decision (e.g. the procedure can be considered as valid or not on the basis of its aptitude to quantify) and those which help to make a diagnosis (e.g. statistical tests evaluating the adequacy of the regression model or the homogeneity of the variances).

In fact, adapted decision tools are really needed to give guarantees that any future measurements will be reasonably inside the acceptance limits. If the guarantees offered by the decision rule are not satisfactory, then the diagnosis tools will help the analyst to identify the possible causes of the problem; but only if the guarantees are not satisfied.

Decision rules

The examination of the current situation with respect to the decision rules used in the validation phase shows that the most of them are based on the use of the null hypothesis as follows.

    H_0 : \text{bias} = 0   \leftrightarrow  H_0 : \text{relative bias} = 0 \%  \leftrightarrow  H_0 : \text{Recouvrement} = 100 \%

with $\text{biais} = x-\mu_T$, the $\text{relative bias} = \frac{x - \mu_T }{\mu_T} \cdot 100$ ans the $\text{recovery} = \frac{ x}{\mu_T} \cdot 100$

On this basis, a procedure is wrongly declared adequate when the 95% confidence interval of the average bias includes the value of 0 (0% and 100% in the case of the relative bias and recovery, respectively). However, this test is inadequate in the validation context of analytical procedures because the decision is based on the computation of the rejection criterion of the Student's t-test. It does not take into account the purpose of the analytical method for determining the acceptance limit and secondly, we note that two conditions contribute to incorrectly validate a method: cases where the standard deviation is large (strong variability where bias will be non significant) and a small number of measurements of repetitions.

Working with an acceptance limit $\pm \lambda$ allows overcoming theses contradiction : reject a method in the case of a low standard deviation and low bias but also to accept a method in which the standard deviation is high and with a large bias. The acceptance criterion is a compromise between bias and variability so that a given proportion of measures, chosen a priori, is in the range $\pm \lambda$.

This latter decision rule appears clearly more sensible than the previous one based on the null hypothesis since all procedures having a small dispersion of the measurements are accepted, while the procedures having a large variance are rejected. In addition, if a procedure has a bias, it should have a small variance to be accepted. Symmetrically, a procedure with a high variance should have a small bias to be accepted.

Note that the use of the null hypothesis, is still widely used in many cases, such as the test of null intercept, of equality of slopes, lack of fit,etc. With all these statistical tests, the less the procedure is precise, the more chance to pass successfully these tests. This situation is certainly not the one expected by the analyst using the statistics to evaluate the capability of the method under investigation.

In this context, a simple and visual decision rule consists of using an accuracy profile with relative acceptance limits $\pm \lambda$.

Accuracy profile computed with SOSstat VPAQ

The accuracy profile, constructed from the confidence intervals on the expected measurements (or statistical dispersion interval), is used to decide whether or not an analysis procedure is capable of giving results within the acceptance limits. The accuracy profile describes the range within which the procedure is able to quantify with a known precision and a risk set a priori by the analyst. For an assumed risk of 5%, for example, the analyst can guarantee that 95 times out of 100 the future measurements given by his procedure will be included within the acceptance limits set according to regulatory requirements (for example: 1% or 2% in volume, 5% on pharmaceutical specialities, 15% in bioanalysis, environment, etc.). The profile is constructed from estimates of the bias and precision of the analytical procedure, as well as the confidence interval, for each concentration level at the end of the validation phase. This confidence interval is also known as the $\beta$ -expectation tolerance interval. It defines the interval within which the expected proportion of future results will fall. This tolerance interval obeys the following property:

    E\{ P[|x_i - \mu_T| < \lambda ]/ \mu_M , \sigma_M   \} \geq \beta

with E meaning expected value of the result.

The calculation of the confidence interval or $\beta$ -expectation tolerance interval requires the estimation of the bias and the intermediate standard deviation of fidelity respectively noted $\mu_M$ and $\sigma_M$.

The accuracy profile can simply be obtained by connecting the lower limits of tolerance or the upper limits of tolerance. If the tolerance interval is larger than the acceptance limits, new limits of quantification and a new dosage interval have to be defined : namely the upper limits of quantification (ULQ) , and the lower limit of quantification (LLQ). The latter is in perfect agreement with the definition of this criterion, i.e. the smallest quantity of the substance to analyze that can be measured with accuracy and a precision defined.

As can be seen, the use of the accuracy profile as single decision tool allows not only to reconcile the objectives of the procedure with those of the validation but also to visually grasp the capacity of the analytical procedure to fit its purpose.

Definitions of ICH

Analytical procedure

The analytical procedure refers to the way of performing the analysis. It should describe in detail the steps necessary to perform each analytical test. This may include but is not limited to: the sample, the reference standard and the reagents preparations, use of the apparatus, generation of the calibration curve, use of the formulae for the calculation, etc.

Specificity

Specificity is the ability to assess unequivocally the analyte in the presence of components which may be expected to be present. Typically these might include impurities, degradants, matrix, etc. Lack of specificity of an individual analytical procedure may be compensated by other supporting analytical procedure(s).

Accuracy

The accuracy of an analytical procedure expresses the closeness of agreement between the value which is accepted either as a conventional true value or an accepted reference value and the value found. This is sometimes termed trueness.

Precision

The precision of an analytical procedure expresses the closeness of agreement (degree of scatter) between a series of measurements obtained from multiple sampling of the same homogeneous sample under the prescribed conditions. Precision may be considered at three levels: repeatability, intermediate precision and reproducibility.

• Repeatability : Repeatability expresses the precision under the same operating conditions over a short interval of time. Repeatability is also termed intra-assay precision .
• Intermediate precision :Intermediate precision expresses within-laboratories variations: different days, different analysts, different equipment, etc.
• Reproducibility : Reproducibility expresses the precision between laboratories (collaborative studies, usually applied to standardization of methodology).

Detection limit

The detection limit of an individual analytical procedure is the lowest amount of analyte in a sample which can be detected but not necessarily quantitated as an exact value.

Quantitation limit

The quantitation limit of an individual analytical procedure is the lowest amount of analyte in a sample which can be quantitatively determined with suitable precision and accuracy. The quantitation limit is a parameter of quantitative assays for low levels of compounds in sample matrices, and is used particularly for the determination of impurities and/or degradation products.

Linearity

The linearity of an analytical procedure is its ability (within a given range) to obtain test results which are directly proportional to the concentration (amount) of analyte in the sample.

Range

The range of an analytical procedure is the interval between the upper and lower concentration (amounts) of analyte in the sample (including these concentrations) for which it has been demonstrated that the analytical procedure has a suitable level of precision, accuracy and linearity.

Robustess

The robustness of an analytical procedure is a measure of its capacity to remain unaffected by small, but deliberate variations in method parameters and provides an indication of its reliability during normal usage.