Search

Deep Neural Networks (DNNs) usually work in an end-to-end manner. This makes the trained DNNs easy to use, but they remain an ambiguous decision process for every test case. Unfortunately, the interpretability of decisions is crucial in some scenarios, such as medical or financial data mining and decision-making. In this paper, we propose a Tree-Network-Tree (TNT) learning framework for explainable decision-making, where the knowledge is alternately transferred between the tree model and DNNs. Specifically, the proposed TNT learning framework exerts the advantages of different models at different stages: (1) a novel James&ndash, Stein Decision Tree (JSDT) is proposed to generate better knowledge representations for DNNs, especially when the input data are in low-frequency or low-quality, (2) the DNNs output high-performing prediction result from the knowledge embedding inputs and behave as a teacher model for the following tree model, and (3) a novel distillable Gradient Boosted Decision Tree (dGBDT) is proposed to learn interpretable trees from the soft labels and make a comparable prediction as DNNs do. Extensive experiments on various machine learning tasks demonstrated the effectiveness of the proposed method.

Gradient boosting decision tree (GBDT) is one of the most widely used ensemble learning methods in both academia and industry. The core idea of GBDT is to consecutively fit new base learners to the residual errors between true and predicted values. However, GBDT is sensitive to noise in the data, especially label noise, which is very common in real-world data. Besides, due to the strong learning ability of GBDT, it is not easy to address this problem with simple ensemble method such as bagging. Thus we propose a new ensemble model of GBDT, called label aggregation of GBDTs (LA-GBDTs). The key points of our method lie on the random feature subspace and label aggregation, which lead to computational efficiency and robustness to noise. Experiments on several benchmark datasets show that label aggregation of GBDTs outperforms GBDT algorithms consistently in terms of accuracy. In addition, LA-GBDTs work well on datasets with a certain degree of label noise.

Template matching is widely used in many applications such as arbitrary object detection and tracking. In this paper, we propose a highly efficient and robust method for template matching. The state-of-the-art nearest neighbor field (NNF)- based algorithms construct the NNF by searching the nearest neighbor of the target points. Instead, our method utilize the inverse NNF by searching the nearest neighbor of template points, which is more robust than original NNF. As for similarity metric, our method utilizes the diversity on the inverse NNF to calculate a global probability image and makes use of the integral image to calculate the similarity score, which makes our method highly efficient. Furthermore, our method is a non-parametric model and has non-explicit assumptions about data. Experiments on challenging public datasets demonstrate the superiority of our method to state- of-the-art methods both in robustness and efficiency.

The ensemble of Convolutional Neural Networks (CNNs) is known to be more accurate and robust than the component CNNs models. Along with the development of a fast training method, current research has managed to make an effective ensemble of several CNNs models and require no additional training cost. However, when the ensemble size of CNNs is further increased, it is hard to observe a corresponding performance enhancement. According to the generalization capability analysis of CNNs, this phenomenon can be explained by the oversaturation of model capacity and the close correlation among the component CNNs, especially when the CNNs are trained within the same dataset. To address this problem, we propose to train CNNs on re-sampled bootstrap datasets. Extensive experiments demonstrate the bootstrap re-sampling is effective for a large ensemble size (up to 80). Besides, benefiting from the usage of the bootstrap re-sampling technique, we can also have an unbiased estimate of the standard deviation of the ensemble output.