Search

• Nonlinear errors severely interfere with the accuracy of chromatic confocal measurements at the submicron scale. • Nonlinear error stems from the deviation between the wavelength-distance mapping and the polynomial model. • The proposed method can effectively record the nonlinear error in calibration and compensate it in measurement. • The results show that the method can achieve a 40 % accuracy improvement over a 4 mm range. The mechanisms of the nonlinear error, primarily stemming from the deviation of the polynomial model fitting the wavelength-distance mapping relationship in chromatic confocal measurement, have been elucidated. A compensation method is proposed for the nonlinear error, based on a regression tree model. Through calibration experiments using our chromatic confocal sensor, an impressive improvement of over 40 % in precision could be achieved by our proposed compensation method, which advances from ±1 μm to ±0.6 μm within a 4 mm measurement range. [ABSTRACT FROM AUTHOR]