Tunable Discounting Mechanisms for Language Modeling

Language models are fundamental to many applications in natural language processing. Most language models are trained on training data that do not support discount parameters tuning. In this work, we present novel language models based on tunable discounting mechanisms. The language models are trained on a large training set, but their discount parameters can be tuned to a target set. We explore tunable discounting and polynomial discounting based on modified Kneser-Ney models. With the resulting implementation, our language models achieve perplexity improvements in in-domain and out-of-domain evaluation. The experimental results indicate that our new models significantly outperform the baseline model and are especially suited for domain adaptation.