Optimization of SVR and CatBoost models using metaheuristic algorithms to assess landslide susceptibility

Abstract In this study, a landslide susceptibility assessment is performed by combining two machine learning regression algorithms (MLRA), such as support vector regression (SVR) and categorical boosting (CatBoost), with two population-based optimization algorithms, such as grey wolf optimizer (GWO) and particle swarm optimization (PSO), to evaluate the potential of a relatively new algorithm and the impact that optimization algorithms can have on the performance of regression models. The Kerala state in India has been chosen as the test site due to the large number of recorded incidents in the recent past. The study started with 18 potential predisposing factors, which were reduced to 14 after a multi-approach feature selection technique. Six susceptibility models were implemented and compared using the machine learning algorithms alone and combining each of them with the two optimization algorithms: SVR, CatBoost, SVR-PSO, CatBoost-PSO, SVR-GWO, and CatBoost-GWO. The resulting maps were validated with an independent dataset. The performance rankings, based on the area under the receiver operating characteristic curve (AUC) metric, are as follows: CatBoost-GWO (AUC = 0.910) had the highest performance, followed by CatBoost-PSO (AUC = 0.909), CatBoost (AUC = 0.899), SVR-GWO (AUC = 0.868), SVR-PSO (AUC = 0.858), and SVR (AUC = 0.840). Other validation statistics corroborated these outcomes, and the Friedman and Wilcoxon-signed rank tests verified the statistical significance of the models. Our case study showed that CatBoost outperformed SVR both in case the models were optimized or not; the introduction of optimization algorithms significantly improves the results of machine learning models, with GWO being slightly more effective than PSO. However, optimization cannot drastically alter the results of the model, highlighting the importance of setting up of a rigorous susceptibility model since the early steps of any research.