1. Leveraging Pre-trained Models for Failure Analysis Triplets Generation
- Author
-
Ezukwoke, Kenneth, Hoayek, Anis, Batton-Hubert, Mireille, Boucher, Xavier, Gounet, Pascal, Adrian, Jerome, Breuil, Florent, Laboratoire d'Informatique, de Modélisation et d'Optimisation des Systèmes (LIMOS), Ecole Nationale Supérieure des Mines de St Etienne (ENSM ST-ETIENNE)-Centre National de la Recherche Scientifique (CNRS)-Université Clermont Auvergne (UCA)-Institut national polytechnique Clermont Auvergne (INP Clermont Auvergne), Université Clermont Auvergne (UCA)-Université Clermont Auvergne (UCA), Département Génie mathématique et industriel (FAYOL-ENSMSE), Ecole Nationale Supérieure des Mines de St Etienne (ENSM ST-ETIENNE)-Institut Henri Fayol, École des Mines de Saint-Étienne (Mines Saint-Étienne MSE), Institut Mines-Télécom [Paris] (IMT), Institut Henri Fayol (FAYOL-ENSMSE), Institut Mines-Télécom [Paris] (IMT)-Institut Mines-Télécom [Paris] (IMT), Centre Ingénierie Santé, Saint-Étienne (CIS - MINES), and STMicroelectronics [Grenoble] (ST-GRENOBLE)
- Subjects
FOS: Computer and information sciences ,I.2 ,Computer Science - Machine Learning ,Computer Science - Computation and Language ,G.3 ,I.7 ,[INFO.INFO-MO]Computer Science [cs]/Modeling and Simulation ,Statistics - Applications ,Machine Learning (cs.LG) ,Applications (stat.AP) ,[INFO.INFO-MO] Computer Science [cs]/Modeling and Simulation ,68Txx, 68Uxx ,Computation and Language (cs.CL) - Abstract
Pre-trained Language Models recently gained traction in the Natural Language Processing (NLP) domain for text summarization, generation and question-answering tasks. This stems from the innovation introduced in Transformer models and their overwhelming performance compared with Recurrent Neural Network Models (Long Short Term Memory (LSTM)). In this paper, we leverage the attention mechanism of pre-trained causal language models such as Transformer model for the downstream task of generating Failure Analysis Triplets (FATs) - a sequence of steps for analyzing defected components in the semiconductor industry. We compare different transformer models for this generative task and observe that Generative Pre-trained Transformer 2 (GPT2) outperformed other transformer model for the failure analysis triplet generation (FATG) task. In particular, we observe that GPT2 (trained on 1.5B parameters) outperforms pre-trained BERT, BART and GPT3 by a large margin on ROUGE. Furthermore, we introduce Levenshstein Sequential Evaluation metric (LESE) for better evaluation of the structured FAT data and show that it compares exactly with human judgment than existing metrics., 33 pages, 11 figures, 9 tables
- Published
- 2022