1. Mathematical optimization in classification and regression trees
- Author
-
Carrizosa, Emilio, Molero-Río, Cristina, and Romero Morales, Dolores
- Subjects
Statistics and Probability ,Mathematical optimization ,Information Systems and Management ,Linear programming ,Computer science ,02 engineering and technology ,Management Science and Operations Research ,01 natural sciences ,010104 statistics & probability ,Tree ensembles ,0202 electrical engineering, electronic engineering, information engineering ,Discrete Mathematics and Combinatorics ,Classification and regression trees ,0101 mathematics ,Continuous optimization ,Flexibility (engineering) ,Complex data type ,Original Paper ,Optimization algorithm ,90C30 ,90C11 ,Explainability ,Regression ,Tree (data structure) ,62-07 ,Decision variables ,Modeling and Simulation ,Mixed-integer linear optimization ,020201 artificial intelligence & image processing ,Sparsity ,Continuous nonlinear optimization - Abstract
Classification and regression trees, as well as their variants, are off-the-shelf methods in Machine Learning. In this paper, we review recent contributions within the Continuous Optimization and the Mixed-Integer Linear Optimization paradigms to develop novel formulations in this research area. We compare those in terms of the nature of the decision variables and the constraints required, as well as the optimization algorithms proposed. We illustrate how these powerful formulations enhance the flexibility of tree models, being better suited to incorporate desirable properties such as cost-sensitivity, explainability, and fairness, and to deal with complex data, such as functional data.
- Published
- 2021