1. Group Invariant Dictionary Learning
- Author
-
Yong Sheng Soh
- Subjects
Signal Processing (eess.SP) ,FOS: Computer and information sciences ,Computer Science - Machine Learning ,Computer science ,Computer Science - Information Theory ,Positive-definite matrix ,Synthetic data ,Machine Learning (cs.LG) ,Matrix (mathematics) ,FOS: Electrical engineering, electronic engineering, information engineering ,FOS: Mathematics ,Penalty method ,Electrical Engineering and Systems Science - Signal Processing ,Electrical and Electronic Engineering ,Invariant (mathematics) ,Mathematics - Optimization and Control ,Artificial neural network ,Information Theory (cs.IT) ,Invariant (physics) ,Hermitian matrix ,Toeplitz matrix ,Algebra ,Matrix group ,Optimization and Control (math.OC) ,Signal Processing ,Homogeneous space - Abstract
The dictionary learning problem concerns the task of representing data as sparse linear sums drawn from a smaller collection of basic building blocks. In application domains where such techniques are deployed, we frequently encounter datasets where some form of symmetry or invariance is present. Motivated by this observation, we develop a framework for learning dictionaries for data under the constraint that the collection of basic building blocks remains invariant under such symmetries. Our procedure for learning such dictionaries relies on representing the symmetry as the action of a matrix group acting on the data, and subsequently introducing a convex penalty function so as to induce sparsity with respect to the collection of matrix group elements. Our framework specializes to the convolutional dictionary learning problem when we consider integer shifts. Using properties of positive semidefinite Hermitian Toeplitz matrices, we develop an extension that learns dictionaries that are invariant under continuous shifts. Our numerical experiments on synthetic data and ECG data show that the incorporation of such symmetries as priors are most valuable when the dataset has few data-points, or when the full range of symmetries is inadequately expressed in the dataset., Comment: 30 pages, 23 figures
- Published
- 2021
- Full Text
- View/download PDF