1. Diverse Embedding Neural Network Language Models
- Author
-
Audhkhasi, Kartik, Sethy, Abhinav, and Ramabhadran, Bhuvana
- Subjects
FOS: Computer and information sciences ,Computer Science - Learning ,Computer Science - Computation and Language ,Computer Science - Neural and Evolutionary Computing ,Neural and Evolutionary Computing (cs.NE) ,Computation and Language (cs.CL) ,Machine Learning (cs.LG) - Abstract
We propose Diverse Embedding Neural Network (DENN), a novel architecture for language models (LMs). A DENNLM projects the input word history vector onto multiple diverse low-dimensional sub-spaces instead of a single higher-dimensional sub-space as in conventional feed-forward neural network LMs. We encourage these sub-spaces to be diverse during network training through an augmented loss function. Our language modeling experiments on the Penn Treebank data set show the performance benefit of using a DENNLM., Comment: Under review as workshop contribution at ICLR 2015
- Published
- 2014
- Full Text
- View/download PDF