Back to Search Start Over

SmoothHess: ReLU Network Feature Interactions via Stein's Lemma

Authors :
Torop, Max
Masoomi, Aria
Hill, Davin
Kose, Kivanc
Ioannidis, Stratis
Dy, Jennifer
Publication Year :
2023

Abstract

Several recent methods for interpretability model feature interactions by looking at the Hessian of a neural network. This poses a challenge for ReLU networks, which are piecewise-linear and thus have a zero Hessian almost everywhere. We propose SmoothHess, a method of estimating second-order interactions through Stein's Lemma. In particular, we estimate the Hessian of the network convolved with a Gaussian through an efficient sampling algorithm, requiring only network gradient calls. SmoothHess is applied post-hoc, requires no modifications to the ReLU network architecture, and the extent of smoothing can be controlled explicitly. We provide a non-asymptotic bound on the sample complexity of our estimation procedure. We validate the superior ability of SmoothHess to capture interactions on benchmark datasets and a real-world medical spirometry dataset.<br />Comment: Accepted to NeurIPS 2023 as a conference paper

Details

Database :
arXiv
Publication Type :
Report
Accession number :
edsarx.2311.00858
Document Type :
Working Paper