1. MockingBERT: A Method for Retroactively Adding Resilience to NLP Models
- Author
-
Jezabek, Jan and Singh, Akash
- Subjects
Computer Science - Computation and Language - Abstract
Protecting NLP models against misspellings whether accidental or adversarial has been the object of research interest for the past few years. Existing remediations have typically either compromised accuracy or required full model re-training with each new class of attacks. We propose a novel method of retroactively adding resilience to misspellings to transformer-based NLP models. This robustness can be achieved without the need for re-training of the original NLP model and with only a minimal loss of language understanding performance on inputs without misspellings. Additionally we propose a new efficient approximate method of generating adversarial misspellings, which significantly reduces the cost needed to evaluate a model's resilience to adversarial attacks., Comment: 8 pages (excl. bibiography and appendix), 2 figures The code necessary for reproduction is available at https://github.com/akash13singh/resilient_nlp To be published in Proceedings of the 29th International Conference on Computational Linguistics (COLING 2022)
- Published
- 2022