Abstract and Compare: A Framework for Defining Precision Measures for Automated Process Discovery

Automated process discovery techniques allow us to extract business process models from event logs. The quality of process models discovered by these techniques can be assessed with respect to various quality criteria related to simplicity and accuracy. One of these criteria, namely precision, captures the extent to which the behavior allowed by a discovered process model is observed in the log. While numerous measures of precision have been proposed in the literature, a recent study has shown that none of them fulfils a set of five axioms that capture intuitive properties behind the concept of precision. In addition, several existing precision measures suffer from scalability issues when applied to models discovered from real-life event logs. This paper presents a versatile framework for defining precision measures based on behavior abstractions. The key idea is that a precision measure can be defined by three ingredients: a function that abstracts a process model (e.g. as a transition system), a function that does the same for an event log, and a function that compares the behavior abstraction of the model with that of the log. We show empirically that different instances of this framework allow us to strike different tradeoffs between scalability and sensitivity. We also show that two instances of the framework based on lossless abstraction functions yield a precision measure that fulfils all the above-mentioned axioms.