ATCoPE - Abstractive text comparison using prompt engineering.

Text summarization is the task of reducing the amount of text from a document while keeping the meaning unchanged. Previously statistical extractive metrics have been used to rank summaries however the current summarization models use an abstractive approach. In an abstractive summarization model the main idea from the original document is taken and it is used to generate the summary. Hence in such cases an abstractive measure of comparison is hence needed. An abstractive measure will give better results over extractive measures when contextual meaning of the text bodies are being considered. In this paper a new approach for abstractive text summarization evaluation is proposed which is based on BERT embeddings and Principal Component Analysis (PCA). The pre-trained BERT model is used to generate embeddings for each sentence in the input text, after which PCA is applied for reducing the dimensionality of the embeddings while preserving the most informative features. The L2 distance metric, which is basically Euclidean distance, is used for comparing the similarity of the reduced dimensional embedding vectors. The proposed approach outperforms the ROUGE metric and shows higher scores for summaries which are contextually similar rather than being textually similar. This metric is used to select sentences from the document which is being summarized to create an improved prompt for state-of-the-art text summarization models, like T5 and GPT, to generate more informative summaries. The prompt based generated summary is validated using ROUGE and METEOR metrics. [ABSTRACT FROM AUTHOR]