In order to ensure the accuracy of clinical test data, it is necessary to assess the validity of the test system. Calibration, calibration verification, or/and AMR validation are important metrics for evaluating quantitative clinical testing systems.
Difference in terminology (I'm actually not a big fan of terminology)
Definition in the CLIA final regulations:
Calibration : the process of testing and adjusting an instrument or test system to establish a correlation between the response of the assay and the concentration or amount of a substance detected by the assay procedure
Calibration validation: the process of detecting a substance at a known concentration, in the same way as a patient sample, to confirm that the whole patient test result is correct. substance in the same manner as a patient sample, confirming the calibration of the instrument or assay system within the reportable range of whole patient test results.
Reportable range: The range of the distribution of test results over which the laboratory can establish or verify the accuracy of the test response of an instrument or test system.
The College of American Pathologists (CAP) uses different terminology that is familiar, which is why "calibration verification" is not well known. Simply put
Analytical Measurement Range (AMR): the analytical range obtained directly by a method
Clinical Reportable Range (CRR): the range of an assay that allows for pre-processing such as dilution, concentration, etc.
In principle, AMR is the equivalent of CLIA's reportable range, whereas CRR provides more practical information that takes into account the method's limit of detection and/or the amount of time that the method is validated by dilution of patient specimens. The procedure for validation of diluted patient specimens extends the AMR.
The method for determining the validity of calibration validation is given in "Validation Evaluation of Quantitative Clinical Analytical Calibration" by H.J. Huang et al. and is briefly described for reference.
In 2005, the Ministry of Health Clinical Inspection Center conducted the first nationwide validation of linearity and calibration, providing calibration validation materials with concentrations or activities that cover the reportable range of most analytes, which were distributed to participating laboratories, and the results were summarized and compared with the target values to determine whether the current calibration is stable within the reportable range and the accuracy of the assay results.
Calibration validation materials, items including routine biochemistry, lipids, enzymes, etc., **** 6 copies, 2 measurements per copy, calculate the average value.
1 Regression equation evaluation: take the mean value of the measured value as the Y-axis, the target value as the X-axis, draw a linear regression graph, calculate the regression equation, and analyze the slope of the regression equation with 1 and the intercept with 0 is statistically significant or not (?). How to judge)
2 Percentage difference evaluation: to the percentage difference that (measured value - target value)/target value * 100% for Y, the corresponding target value for X, plot the percentage difference graph, where the low concentration judgment limit is the ratio of the smallest detected difference to the target value, the high concentration judgment limit is the 1/2 percent of the total error analysis target, plot the evaluation graph of the calibration validation and determine whether the points are within the judgment limit. Acceptable total error analysis target and minimum detection difference table (omitted) self-consultation.
1) Validation 1: regression equation evaluation ok, percent difference evaluation ok?
2) Validation 2: regression equation evaluation not ok, percent difference evaluation all ok
3) Difference 1: regression equation evaluation ok, ? However, the calibration validation measurements at least one of the values exceeds the acceptable range, the analytical detection system calibration is invalid.
4) Discrepancy 2: The regression equation evaluated ? Not ok, calibration verification of at least one of the measured values exceeds the acceptable range, the analytical detection system calibration is invalid.
Where Validation 1 and Validation 2? are acceptable results for calibration validation, Discrepancy 1 and Discrepancy 2 are unacceptable.
Discrepancy 1 is most often caused by random error, which requires control of the precision of the experiment. Discrepancy 2 indicates a proportional or constant systematic error in the analytical detection system, which requires methodological confirmation. Laboratory for difference 1 and difference 2 should be investigated and analyzed to take effective measures to improve.
In short, clinical quantitative testing laboratories through the calibration, calibration verification and AMR confirmation process, not only to ensure the stability and accuracy of the detection system, but also to help the laboratory to find the problem, correct the problem, to ensure that the provision of accurate and reliable test results.
One last question: how do you analyze whether the slope of the regression equation is statistically significant from 1 and the intercept from 0?
It was a pain in the ass, and I couldn't find it anywhere. Finally, I found it in an exercise in one of the chapters of "Fundamentals of Confirmation of Medical Laboratory Methods" by Mr. Westgard. Briefly
The slope and standard deviation of the slope, and the intercept and standard deviation of the intercept are known, e.g., Y=bx+a, a=-0.31, s(a)=0.23 , b= 1.032, s(b)=0.009, and the slope is statistically different from 1, and the intercept is statistically different from 0?
The 95% likelihood of the slope b can be calculated from the slope b +/- 2 times the standard deviation, i.e., 1.014 to 1.050. Because this range does not overlap with the ideal slope 1, it can be argued that the difference between the observed and ideal slopes is statistically significant. And by the same token, the intercept 95% likelihood range is -0.76~0.15, and this range includes the ideal value 0 , so there is no significant difference between the intercept and the ideal value.
Also, where do the slopes and the standard deviation of the intercepts come from? I don't get it either, so leave a tail.
Postscript: it is not easy to do a good job of a topic, it seems simple four words of the theme, behind the hidden knowledge of many, many, many, from the concept to the implementation, and then interpreted, it is really hard to think, the sun on the information at hand.
The end of the year and the beginning of the year is often the most work, there are internal audit, management review, higher hospitals for further training, etc., did not update in a timely manner to apologize to all of you, in fact, this should not be the reason, I hope that more support! There is the author's laboratory this year to participate in the Ministry of Health's accuracy verification evaluation, and so the results back to discuss and share it with you.