Back to Search
Start Over
TNT: An Interpretable Tree-Network-Tree Learning Framework using Knowledge Distillation
- Source :
- Entropy, Volume 22, Issue 11, Entropy, Vol 22, Iss 1203, p 1203 (2020)
- Publication Year :
- 2020
- Publisher :
- MDPI, 2020.
-
Abstract
- Deep Neural Networks (DNNs) usually work in an end-to-end manner. This makes the trained DNNs easy to use, but they remain an ambiguous decision process for every test case. Unfortunately, the interpretability of decisions is crucial in some scenarios, such as medical or financial data mining and decision-making. In this paper, we propose a Tree-Network-Tree (TNT) learning framework for explainable decision-making, where the knowledge is alternately transferred between the tree model and DNNs. Specifically, the proposed TNT learning framework exerts the advantages of different models at different stages: (1) a novel James&ndash<br />Stein Decision Tree (JSDT) is proposed to generate better knowledge representations for DNNs, especially when the input data are in low-frequency or low-quality<br />(2) the DNNs output high-performing prediction result from the knowledge embedding inputs and behave as a teacher model for the following tree model<br />and (3) a novel distillable Gradient Boosted Decision Tree (dGBDT) is proposed to learn interpretable trees from the soft labels and make a comparable prediction as DNNs do. Extensive experiments on various machine learning tasks demonstrated the effectiveness of the proposed method.
- Subjects :
- Computer science
Decision tree
James–Stein Decision Trees
General Physics and Astronomy
lcsh:Astrophysics
02 engineering and technology
Machine learning
computer.software_genre
Article
020204 information systems
lcsh:QB460-466
0202 electrical engineering, electronic engineering, information engineering
distillable gradient boosted decision tree
lcsh:Science
Interpretability
business.industry
interpretable machine learning
lcsh:QC1-999
Tree (data structure)
deep neural networks
knowledge distillation
Tree network
Deep neural networks
Embedding
lcsh:Q
020201 artificial intelligence & image processing
Artificial intelligence
Gradient boosting
business
computer
lcsh:Physics
Decision tree model
Subjects
Details
- Language :
- English
- ISSN :
- 10994300
- Volume :
- 22
- Issue :
- 11
- Database :
- OpenAIRE
- Journal :
- Entropy
- Accession number :
- edsair.doi.dedup.....101892f485b64fe4a4d3fd170d0aa51c