Evaluation of Pooling Operations and Regularization Parameters in Neural Networks for Drug-drug Interaction Extraction

Recently, deep neural networks have been widely used in biomedical relation extraction, especially CNN and LSTM. Relation extraction is a classification task which uses pooling operations in neural networks to reduce dimensions and integration features and it uses regularization to prevent over-fitting. This paper evaluates the performance of CNN and LSTM in Drug-drug interaction extraction. We discuss the models' performance differences using different pooling operations and regularization parameters. Firstly, regularization can prevent over-fitting effectively, but it is important to ensure that the regularization parameters are set within the correct ranges. Secondly, the max pooling is better than other single pooling methods. Max pooling outperforms the others alternatives because is the only one which is invariant to the special pad tokens that are appending to the shorter sentences known as padding.