False precision of mass domain in HPLC–HRMS data representation.

The term precision of a value describes the total number of significant digits that are used to express that value. False precision is presented in data, where the sampling and chosen data file format coding return values with extra non-valid digits. The proper precision in value thus depends on so-called magnitude in digital data coding. The presence of the false precision is file format dependent. The amount of dataset points could be decreased by removing false precision (where it is present) to almost 10% of the raw measurement without loss of information. [ABSTRACT FROM AUTHOR]