Search

The popularity of egocentric cameras and their always-on nature has lead to the abundance of day long first-person videos. The highly redundant nature of these videos and extreme camera-shakes make them difficult to watch from beginning to end. These videos require efficient summarization tools for consumption. However, traditional summarization techniques developed for static surveillance videos or highly curated sports videos and movies are either not suitable or simply do not scale for such hours long videos in the wild. On the other hand, specialized summarization techniques developed for egocentric videos limit their focus to important objects and people. This paper presents a novel unsupervised reinforcement learning framework to summarize egocentric videos both in terms of length and the content. The proposed framework facilitates incorporating various prior preferences such as faces, places, or scene diversity and interactive user choice in terms of including or excluding the particular type of content. This approach can also be adapted to generate summaries of various lengths, making it possible to view even 1-minute summaries of one's entire day. When using the facial saliency-based reward, we show that our approach generates summaries focusing on social interactions, similar to the current state-of-the-art (SOTA). The quantitative comparisons on the benchmark Disney dataset show that our method achieves significant improvement in Relaxed F-Score (RFS) (29.60 compared to 19.21 from SOTA), BLEU score (0.68 compared to 0.67 from SOTA), Average Human Ranking (AHR), and unique events covered. Finally, we show that our technique can be applied to summarize traditional, short, hand-held videos as well, where we improve the SOTA F-score on benchmark SumMe and TVSum datasets from 41.4 to 46.40 and 57.6 to 58.3 respectively. We also provide a Pytorch implementation and a web demo at https://pravin74.github.io/Int-sum/index.html.

Data provenance is any information about the origin of a piece of data and the process that led to its creation. Most database provenance work has focused on creating models and semantics to query and generate this information. While comprehensive, provenance information remains large and overwhelming, which can make it hard for provenance systems to support data exploration. We present a new approach to provenance exploration that builds on data summarization techniques. We contribute two novel summarization schemes for the provenance of aggregation queries: Impact summaries, and comparative summaries. We show with experiments that our techniques incur little overhead compared to basic summaries. We conduct a survey to show that our approaches are useful to users.

Feedback is important to improve writing quality; however, to provide timely and personalized feedback is a time-intensive task. Currently, most literature focuses on providing (human or machine) support on product characteristics, especially after a draft is submitted. However, this does not assist students who struggle during the writing process. Therefore, in this study, we investigate the use of keystroke analysis to predict writing quality throughout the writing process. Keystroke data were analyzed from 126 English as a second language learners performing a timed academic summarization task. Writing quality was measured using participants’ final grade. Based on previous literature, 54 keystroke features were extracted. Correlational analyses were conducted to identify the relationship between keystroke features and writing quality. Next, machine learning models (regression and classification) were used to predict final grade and classify students who might need support at several points during the writing process. The results show that, in contrast to previous work, the relationship between writing quality and keystroke data was rather limited. None of the regression models outperformed the baseline, and the classification models were only slightly better than the majority class baseline (highest AUC = 0.57). In addition, the relationship between keystroke features and writing quality changed throughout the course of the writing process. To conclude, the relationship between keystroke data and writing quality might be less clear than previously posited.

Short text clustering has far-reaching effects on semantic analysis, showing its importance for multiple applications such as corpus summarization and information retrieval. However, it inevitably encounters the severe sparsity of short text representations, making the previous clustering approaches still far from satisfactory. In this paper, we present a novel attentive representation learning model for shot text clustering, wherein cluster-level attention is proposed to capture the correlations between text representations and cluster representations. Relying on this, the representation learning and clustering for short texts are seamlessly integrated into a unified model. To further ensure robust model training for short texts, we apply adversarial training to the unsupervised clustering setting, by injecting perturbations into the cluster representations. The model parameters and perturbations are optimized alternately through a minimax game. Extensive experiments on four real-world short text datasets demonstrate the superiority of the proposed model over several strong competitors, verifying that robust adversarial training yields substantial performance gains., Comment: 14pages, to appear in IEEE TKDE

In this paper, we propose a novel deep Efficient Relational Sentence Ordering Network (referred to as ERSON) by leveraging pre-trained language model in both encoder and decoder architectures to strengthen the coherence modeling of the entire model. Specifically, we first introduce a divide-and-fuse BERT (referred to as DF-BERT), a new refactor of BERT network, where lower layers in the improved model encode each sentence in the paragraph independently, which are shared by different sentence pairs, and the higher layers learn the cross-attention between sentence pairs jointly. It enables us to capture the semantic concepts and contextual information between the sentences of the paragraph, while significantly reducing the runtime and memory consumption without sacrificing the model performance. Besides, a Relational Pointer Decoder (referred to as RPD) is developed, which utilizes the pre-trained Next Sentence Prediction (NSP) task of BERT to capture the useful relative ordering information between sentences to enhance the order predictions. In addition, a variety of knowledge distillation based losses are added as auxiliary supervision to further improve the ordering performance. The extensive evaluations on Sentence Ordering, Order Discrimination, and Multi-Document Summarization tasks show the superiority of ERSON to the state-of-the-art ordering methods.

Logs contain valuable information about the runtime behaviors of software systems. Thus, practitioners rely on logs for various tasks such as debugging, system comprehension, and anomaly detection. However, due to the unstructured nature and large size of logs, there are several challenges that practitioners face with log analysis. In this paper, we propose a novel approach called LogAssist that tackles these challenges and assists practitioners with log analysis. LogAssist provides an organized and concise view of logs by first grouping logs into event sequences (i.e., workflows), which better illustrate the system runtime execution paths. Then, LogAssist compresses the log events in workflows by hiding consecutive events and applying n-gram modeling to identify common event sequences. We evaluated LogAssist on the logs that are generated by two open-source and one enterprise system. We find that LogAssist can reduce the number of log events that practitioners need to investigate by up to 99%. Through a user study with 19 participants, we also find that LogAssist can assist practitioners by reducing the needed time on log analysis tasks by an average of 40%. The participants also rated LogAssist an average of 4.53 out of 5 for improving their experiences of performing log analysis. Finally, we document our experiences and lessons learned from developing and adopting LogAssist in practice. We believe that LogAssist and our reported experiences may lay the basis for future analysis and interactive exploration on logs.

Extracting relevant information from a large amount of data is a challenging task. Automatic text summarization is a potential solution for obtaining this information. In this paper, a nature inspired swarm intelligence-based algorithm viz. firefly algorithm for multi-document text summarization is proposed. A new fitness function consisting of three features viz. topic relation factor, cohesion factor and readability factor is utilized. The experiments are performed on datasets from Document Understanding Conference i.e. DUC-2002, DUC-2003 and DUC-2004. The performance of the algorithm has been evaluated using ROUGE score. The performance of the proposed algorithm is compared with some other nature inspired ones such as particle swarm optimization (PSO) and genetic algorithm (GA). The performance of the proposed algorithm outperforms the other adopted ones.

In the natural language processing family, learning representations is a pioneering study, especially in sequence-to-sequence tasks where outputs are generated, totally relying on the learning representations of source sequence. Generally, classic methods infer that each word occurring in the source sequence, having more or less influence on the target sequence, should all be considered when generating outputs. As the summarization task requires the output sequence to only retain the essence, classic full consideration of the source sequence may not work well on it, which calls for more suitable methods with the ability to discard the misleading noise words. Motivated by this, with both relevance retaining and redundancy removal in mind, we propose a summarization learning model by implementing an encoder with copious contextual information represented and a decoder with a selecting mechanism integrated. Specifically, we equip the encoder with an asynchronous bi directional parallel structure, in order to obtain abundant semantic representation. The decoder, different from the classic attention-based works, employs a self-aware context selecting mechanism to generate summary in a more productive way. We evaluate the proposed methods on three benchmark summarization corpora. The experimental results demonstrate the effectiveness and applicability of the proposed framework in relation to several well-practiced and state-of-the-art summarization methods.

The day-to-day growth of online information necessitates intensive research in automatic text summarization (ATS). The ATS software produces summary text by extracting important information from the original text. With the help of summaries, users can easily read and understand the documents of interest. Most of the approaches for ATS used only local properties of text. Moreover, the numerous properties make the sentence selection difficult and complicated. So this article uses a graph based summarization to utilize structural and global properties of text. It introduces maximal clique based sentence selection (MCBSS) algorithm to select important and non-redundant sentences that cover all concepts of the input text for summary. The MCBSS algorithm finds novel information using maximal cliques (MCs). The experimental results of recall oriented understudy for gisting evaluation (ROUGE) on Timeline dataset show that the proposed work outperforms the existing graph algorithms Bushy Path (BP), Aggregate Similarity (AS), and TextRank (TR).

In the era of social networks, the rapid growth of data mining in information retrieval and natural language processing makes automatic text summarization necessary. Currently, pretrained word embedding and sequence to sequence models can be effectively adapted in social network summarization to extract significant information with strong encoding capability. However, how to tackle the long text dependence and utilize the latent topic mapping has become an increasingly crucial challenge for these models. In this article, we propose a topic-aware extractive and abstractive summarization model named T-BERTSum, based on Bidirectional Encoder Representations from Transformers (BERTs). This is an improvement over previous models, in which the proposed approach can simultaneously infer topics and generate summarization from social texts. First, the encoded latent topic representation, through the neural topic model (NTM), is matched with the embedded representation of BERT, to guide the generation with the topic. Second, the long-term dependencies are learned through the transformer network to jointly explore topic inference and text summarization in an end-to-end manner. Third, the long short-term memory (LSTM) network layers are stacked on the extractive model to capture sequence timing information, and the effective information is further filtered on the abstractive model through a gated network. In addition, a two-stage extractive-abstractive model is constructed to share the information. Compared with the previous work, the proposed model T-BERTSum focuses on pretrained external knowledge and topic mining to capture more accurate contextual representations. Experimental results on the CNN/Daily mail and XSum datasets demonstrate that our proposed model achieves new state-of-the-art results while generating consistent topics compared with the most advanced method.

Update summarization is a relatively recent summarization task concerned with creating a short summary from news articles, assuming the user has already read a number of previous articles. It is useful for users who want to know the latest information about a specific topic. The availability of systems that provide update summaries saves user time and effort. Unfortunately, such resources are lacking for the Arabic language. This paper aims to provide an update summarization system that generates an update summary containing the latest information requested by a user from multiple documents. This paper provides an Arabic query-based Update-Summarization System (AUSS). We use a graph-based ranking model to represent the similarity, through a combination of lexical and semantic relations between words. Our experiments show that AUSS achieves promising results, achieving a best F-Measure of 0.5405. To test AUSS, we created a new corpus especially for Arabic Update Summaries, which we call (AUS-DB). AUS-DB contains 183 articles and their corresponding reference summaries.

The availability of legal judgment documents in digital form offers numerous opportunities for information extraction and application. Automatic summarization of these legal texts is a crucial and a challenging task due to the unusual structure and high complexity of these documents. Previous approaches in this direction have relied on huge labelled datasets, using hand engineered features, leveraging on domain knowledge and focussed their attention on a narrow sub-domain for increased effectiveness. In this paper, we propose simple generic techniques using neural network for the summarization task for Indian legal judgment documents. We explore two neural network architectures for this task utilizing the word and sentence embeddings for capturing the semantics. The main advantage of the proposed approaches is that they do not rely on hand crafted features, or domain specific knowledge, nor is their application restricted to a particular sub-domain thus making them suitable to be extended to other domains as well. We tackle the problem of unavailability of labelled data for the task by assigning classes/scores to sentences in the training set, based on their match with reference summary produced by humans. The experimental evaluations establish the effectiveness of our proposed approaches as compared with other baselines.

Document clustering has recently been paid great attention in retrieval, navigation, and summarization of huge volumes of documents. With a better document clustering approach, computers can organize a document corpus automatically to a meaningful cluster for enabling efficient navigation, and browsing of the corpus. Document navigation and browsing is a valuable complement to the deficiencies of information retrieval technologies. This paper introduces Modsup-based frequent itemset and Rider Optimization-based Moth Search Algorithm (Rn-MSA) for clustering the documents. At first, the input documents are given to the pre-processing step, and then, the extraction is carried out based on TF-IDF and Wordnet features. Once the extraction is done, the feature selection is carried out based on frequent itemset for the establishment of feature knowledge. At last, the document clustering is done using the proposed Rn-MSA, which is designed by combining Rider Optimization Algorithm (ROA), and the Moth Search Algorithm (MSA). The performance of the document clustering based on proposed Modsup + Rn-MSA is evaluated in terms of precision, recall, F-Measure, and accuracy. The developed document clustering method achieves the maximal precision of 95.90%, maximal recall of 96.41%, maximal F-Measure of 96.41%, and the maximal accuracy of 95.12% that indicates its superiority.

Text summarization automatically produces a summary containing important sentences and includes all relevant important information from the original document. One of the main approaches, when viewed from the summary results, are extractive and abstractive. An extractive summary is heading towards maturity and now research has shifted towards abstractive summation and real-time summarization. Although there have been so many achievements in the acquisition of datasets, methods, and techniques published, there are not many papers that can provide a broad picture of the current state of research in this field. This paper provides a broad and systematic review of research in the field of text summarization published from 2008 to 2019. There are 85 journal and conference publications which are the results of the extraction of selected studies for identification and analysis to describe research topics/trends, datasets, preprocessing, features, techniques, methods, evaluations, and problems in this field of research. The results of the analysis provide an in-depth explanation of the topics/trends that are the focus of their research in the field of text summarization; provide references to public datasets, preprocessing and features that have been used; describes the techniques and methods that are often used by researchers as a comparison and means for developing methods. At the end of this paper, several recommendations for opportunities and challenges related to text summarization research are mentioned.

The scientific research process generally starts with the examination of the state of the art, which may involve a vast number of publications. Automatically summarizing scientific articles would help researchers in their investigation by speeding up the research process. The automatic summarization of scientific articles differs from the summarization of generic texts due to their specific structure and inclusion of citation sentences. Most of the valuable information in scientific articles is presented in tables, figures, and algorithm pseudocode. These elements, however, do not usually appear in a generic text. Therefore, several approaches that consider the particularity of a scientific article structure were proposed to enhance the quality of the generated summary, resulting in ad hoc automatic summarizers. This paper provides a comprehensive study of the state of the art in this field and discusses some future research directions. It particularly presents a review of approaches developed during the last decade, the corpora used, and their evaluation methods. It also discusses their limitations and points out some open problems. The conclusions of this study highlight the prevalence of extractive techniques for the automatic summarization of single monolingual articles using a combination of statistical, natural language processing, and machine learning techniques. The absence of benchmark corpora and gold standard summaries for scientific articles remains the main issue for this task.

Extraction of knowledge from soccer videos has enormous applications like context-based advertisement, content-based video retrieval, match summarization, and highlight extraction. Overlapping soccer actions and uncontrolled video capturing conditions make it challenging to detect action accurately. For overcoming these problems, Convolutional Neural Network and Recurrent Neural Network are used jointly to classify different lengths of soccer actions. Initially, transfer learning from pre-trained VGG network extracts characteristic spatial features. Afterwards, Gated Recurrent Unit deals with temporal dependency and solves the vanishing gradient problem. Finally, softmax layer assigns decimal probabilities to each class. Experimental results demystify the significance of the proposed architecture.

With the explosive growth of videos captured everyday, how to efficiently extract useful information from videos has become a more and more important problem. As one of the most effective methods, video summarization aiming to extract the most important frames or shots has attracted more interests recently. Currently, lots of methods employ a recurrent structure. However, due to its step-by-step characteristic, it is difficult to parallelize these models. To address this problem, we propose a dual-path attentive video summarization framework consisting of a temporal spatial encoder, a score-aware encoder and a decoder. And all of them are mainly based on multi-head self-attention and convolutional block attention module. The temporal spatial encoder is to capture the temporal and spatial information while the score-aware encoder incorporates the appearance features with previously predicted frame-level importance scores. By combining the scores and appearance features, our model can better capture the long-range global dependencies and update the importance scores of previous frames continuously. Moreover, entirely based on attention mechanism, our model can be trained in full parallel, which leads to less training time. To validate the method, we employ the two popular datasets SumMe and TVSum. The experimental results show the effectiveness of the proposed method.

Although video summarization has achieved tremendous success benefiting from Recurrent Neural Networks (RNN), RNN-based methods neglect the global dependencies and multi-hop relationships among video frames, which limits the performance. Transformer is an effective model to deal with this problem, and surpasses RNN-based methods in several sequence modeling tasks, such as machine translation, video captioning, etc. Motivated by the great success of transformer and the natural structure of video (frame-shot-video), a hierarchical transformer is developed for video summarization, which can capture the dependencies among frame and shots, and summarize the video by exploiting the scene information formed by shots. Furthermore, we argue that both the audio and visual information are essential for the video summarization task. To integrate the two kinds of information, they are encoded in a two-stream scheme, and a multimodal fusion mechanism is developed based on the hierarchical transformer. In this paper, the proposed method is denoted as Hierarchical Multimodal Transformer (HMT). Practically, extensive experiments show that HMT achieves (F-measure: 0.441, Kendall’s τ : 0.079, Spearman’s ρ : 0.080) and (F-measure: 0.601, Kendall’s τ : 0.096, Spearman’s ρ : 0.107) on SumMe and TVsum, respectively. It surpasses most of the traditional, RNN-based and attention-based video summarization methods.

Ever since the first introduction of Artificial Intelligence into the field of hydrology, it has further generated immense interest in researching aspects for further improvements to hydrology. This can be seen in the rising number of related works published. This culminated further with the combination of pioneering optimization techniques. Who would have thought that the birds and the bees can offer advances in the mathematical sciences and so have the ants too? The ingenuity of humans is spelled out in the algorithms that mimic many natural activities, like pack hunting by the wolves! This review paper serves to broadcast more of the intriguing interest in newfound procedures in optimal forecasting. Reservoirs are the main and most efficient water storage facilities for managing uneven water distribution. However, due to the major global climate changes which affect rainfall trend and weather, it has been a necessity to find an alternative solution for effective conventional water balance. A multifunctional reservoir operation appears to require the operator to make wise decisions to achieve an optimal reservoir operation. One of the most important aspects of all this is the forecasting of streamflows. For this, Artificial Intelligence (AI) seems to be the best alternative solution; as in the past three decades, there has been a drastic increase in building and developing AI models for forecasting and modelling unstable patterns in various hydrological fields. Nevertheless, AI models are also required to be optimized in tandem to achieve the best result, leading thus to the desirous forming of hybrid models between a standalone AI model and optimization techniques. This comprehensive study categorizes machine learning into three main categories, together with the optimization techniques, and will next explore the various AI model used for different hydrology fields along with the most common optimization techniques. Summarization of findings under every section is provided. Some advantages and disadvantages found through literature reviews are summarized for ease of reference. Finally, future recommendations and overall conclusions drawn from the results of researchers are included. This current review focuses on papers from high-impact factor publications based on 10 years starting from (2009 to 2020).

3D printing or additive manufacturing process works on the stereolithographic process. It is based on the x, y and z-axis whereas in 4D printing another dimension is added which is time and with the combination of smart material alteration in shape, properties and function are possible under the corresponding external stimulus. The shape-shifting behaviour of 4D printed objects gives it an edge over 3D printing but it has a limitation of producing objects having curved layers and requiring higher strength. The concepts and applicability of 5D printing (5DP) are discussed. Summarization of the 3D and 4D printing process along with their limitations are also presented. The limitation of 3D and 4D printing of not printing those structures or complex shapes which require higher strength or have a curve layer can be printed with 5D printing. In the case of 5DP, there are two additional axes i.e. rotation of print bed and rotation of extruder head. In this study detailed process, general application and a case study of 5DP in the healthcare and medical industry focussed on dental surgery and orthopaedics is summarized.

Code summarization (aka comment generation) provides a high-level natural language description of the function performed by code, which can benefit the software maintenance, code categorization and retrieval. To the best of our knowledge, the state-of-the-art approaches follow an encoder-decoder framework which encodes source code into a hidden space and later decodes it into a natural language space. Such approaches suffer from the following drawbacks: (a) they are mainly input by representing code as a sequence of tokens while ignoring code hierarchy; (b) most of the encoders only input simple features (e.g., tokens) while ignoring the features that can help capture the correlations between comments and code; (c) the decoders are typically trained to predict subsequent words by maximizing the likelihood of subsequent ground truth words, while in real world, they are excepted to generate the entire word sequence from scratch. As a result, such drawbacks lead to inferior and inconsistent comment generation accuracy. To address the above limitations, this paper presents a new code summarization approach using hierarchical attention network by incorporating multiple code features, including type-augmented abstract syntax trees and program control flows. Such features, along with plain code sequences, are injected into a deep reinforcement learning (DRL) framework (e.g., actor-critic network) for comment generation. Our approach assigns weights (pays “attention”) to tokens and statements when constructing the code representation to reflect the hierarchical code structure under different contexts regarding code features (e.g., control flows and abstract syntax trees). Our reinforcement learning mechanism further strengthens the prediction results through the actor network and the critic network, where the actor network provides the confidence of predicting subsequent words based on the current state, and the critic network computes the reward values of all the possible extensions of the current state to provide global guidance for explorations. Eventually, we employ an advantage reward to train both networks and conduct a set of experiments on a real-world dataset. The experimental results demonstrate that our approach outperforms the baselines by around 22% to 45% in BLEU-1 and outperforms the state-of-the-art approaches by around 5% to 60% in terms of S-BLEU and C-BLEU.

Semantic segmentation is a critical component in autonomous driving and has to be thoroughly evaluated due to safety concerns. Deep neural network (DNN) based semantic segmentation models are widely used in autonomous driving. However, it is challenging to evaluate DNN-based models due to their black-box-like nature, and it is even more difficult to assess model performance for crucial objects, such as lost cargos and pedestrians, in autonomous driving applications. In this work, we propose VASS, a Visual Analytics approach to diagnosing and improving the accuracy and robustness of Semantic Segmentation models, especially for critical objects moving in various driving scenes. The key component of our approach is a context-aware spatial representation learning that extracts important spatial information of objects, such as position, size, and aspect ratio, with respect to given scene contexts. Based on this spatial representation, we first use it to create visual summarization to analyze models' performance. We then use it to guide the generation of adversarial examples to evaluate models' spatial robustness and obtain actionable insights. We demonstrate the effectiveness of VASS via two case studies of lost cargo detection and pedestrian detection in autonomous driving. For both cases, we show quantitative evaluation on the improvement of models' performance with actionable insights obtained from VASS.

Video moderation, which refers to remove deviant or explicit content from e-commerce livestreams, has become prevalent owing to social and engaging features. However, this task is tedious and time consuming due to the difficulties associated with watching and reviewing multimodal video content, including video frames and audio clips. To ensure effective video moderation, we propose VideoModerator, a risk-aware framework that seamlessly integrates human knowledge with machine insights. This framework incorporates a set of advanced machine learning models to extract the risk-aware features from multimodal video content and discover potentially deviant videos. Moreover, this framework introduces an interactive visualization interface with three views, namely, a video view, a frame view, and an audio view. In the video view, we adopt a segmented timeline and highlight high-risk periods that may contain deviant information. In the frame view, we present a novel visual summarization method that combines risk-aware features and video context to enable quick video navigation. In the audio view, we employ a storyline-based design to provide a multi-faceted overview which can be used to explore audio content. Furthermore, we report the usage of VideoModerator through a case scenario and conduct experiments and a controlled user study to validate its effectiveness., 9 pages, 2 references

Building a visual overview of temporal event sequences with an optimal level-of-detail (i.e. simplified but informative) is an ongoing challenge - expecting the user to zoom into every important aspect of the overview can lead to missing insights. We propose a technique to build a multilevel overview of event sequences, whose granularity can be transformed across sequence clusters (vertical level-of-detail) or longitudinally (horizontal level-of-detail), using hierarchical aggregation and a novel cluster data representation Align-Score-Simplify. By default, the overview shows an optimal number of sequence clusters obtained through the average silhouette width metric - then users are able to explore alternative optimal sequence clusterings. The vertical level-of-detail of the overview changes along with the number of clusters, whilst the horizontal level-of-detail refers to the level of summarization applied to each cluster representation. The proposed technique has been implemented into a visualization system called Sequence Cluster Explorer (Sequen-C) that allows multilevel and detail-on-demand exploration through three coordinated views, and the inspection of data attributes at cluster, unique sequence, and individual sequence level. We present two case studies using real-world datasets in the healthcare domain: CUREd and MIMIC-III; which demonstrate how the technique can aid users to obtain a summary of common and deviating pathways, and explore data attributes for selected patterns., Comment: This is the author's version of the article to be published in IEEE Transactions on Visualization and Computer Graphics

Due to recent advances in the film industry, the production of movies has grown exponentially, which has led to challenges in what is referred to as discoverability: given the overwhelming number of choices, choosing which film to watch has become a tedious task for audiences. Movie summarization (MS) could help, as it presents the central theme of the movie in a compact format and makes browsing more efficient for the audience. In this paper, we present an automatic MS framework coined as ‘QuickLook’, which identifies the leading characters and fuses multiple cues extracted from a movie. Firstly, the movie data is preprocessed for its division into scenes, followed by shot segmentation. Secondly, the leading characters in each segmented scene are determined. Next, four visual cues that capture the film's scenic beauty, memorability, informativeness and emotional resonance are extracted from shots containing the leading characters. These extracted features are then intelligently fused based on the assignment of different weights; shots with a fusion score above a certain threshold are selected for the final summary. The proposed MS framework is assessed by comparison with official trailers from ten Hollywood movies, providing a novel baseline for future fair comparison in the MS literature. The proposed framework is shown to outperform other state-of-the-art MS methods in terms of enjoyability and informativeness.

Aggregation functions, which are at the heart of a number of information fusion processes, allow summarization of multiple inputs into a single representative value. Extended aggregation functions are defined such that the input data can be of varying cardinality, with the implication that there is some consistency across the methods of calculation. This article formalizes an approach to extended aggregation such that contributions of repeated inputs or regions of high density are limited in their ability to influence the final value. We establish important definitions and properties, in particular around whether such functions will be monotone or directionally monotone. We then propose a powerful construction method for extended pre-aggregation functions. Illustrative examples are provided throughout.

Nowadays, most research conducted in the field of abstractive text summarization focuses on neural-based models alone, without considering their combination with knowledge-based approaches that could further enhance their efficiency. In this direction, this work presents a novel framework that combines sequence-to-sequence neural-based text summarization along with structure and semantic-based methodologies. The proposed framework is capable of dealing with the problem of out-of-vocabulary or rare words, improving the performance of the deep learning models. The overall methodology is based on a well-defined theoretical model of knowledge-based content generalization and deep learning predictions for generating abstractive summaries. The framework is composed of three key elements: (i) a pre-processing task, (ii) a machine learning methodology, and (iii) a post-processing task. The pre-processing task is a knowledge-based approach, based on ontological knowledge resources, word sense disambiguation, and named entity recognition, along with content generalization, that transforms ordinary text into a generalized form. A deep learning model of attentive encoder-decoder architecture, which is expanded to enable a coping and coverage mechanism, as well as reinforcement learning and transformer-based architectures, is trained on a generalized version of text-summary pairs, learning to predict summaries in a generalized form. The post-processing task utilizes knowledge resources, word embeddings, word sense disambiguation, and heuristic algorithms based on text similarity methods in order to transform the generalized version of a predicted summary to a final, human-readable form. An extensive experimental procedure on three popular data sets evaluates key aspects of the proposed framework, while the obtained results exhibit promising performance, validating the robustness of the proposed approach.

Purpose This paper aims to focus on the trends and projection for future use of artificial intelligence (AI) in libraries. AI technologies is the latest among the technologies being used in libraries. The technology has systems that have natural language processing, machine learning and pattern recognition capabilities that make service provision easier for libraries. Design/methodology/approach Systematic literature review is done, exploring blogs and wikis, to collect information on the ways in which AI is used and can be futuristically used in libraries. Findings This paper found that uses of AI in libraries entailed enhanced services such as content indexing, document matching, content mapping content summarization and many others. AI possibilities were also found to include improving the technology of gripping, localizing and human–robot interaction and also having artificial superintelligence, the hypothetical AI that surpasses human intelligence and abilities. Originality/value It is concluded that advanced technologies that AI are, will help librarians to open up new horizons and solve challenges that crop up in library service delivery.

The standard approach for abstractive text summarization is to use an encoder-decoder architecture. The encoder is responsible for capturing the general meaning from the source text, and the decoder is in charge of generating the final text summary. While this approach can compose summaries that resemble human writing, some may contain unrelated or unfaithful information. This problem is called “hallucination” and it represents a serious issue in legal texts as legal practitioners rely on these summaries when looking for precedents, used to support legal arguments. Another concern is that legal documents tend to be very long and may not be fed entirely to the encoder. We propose our method called LegalSumm for addressing these issues by creating different “views” over the source text, training summarization models to generate independent versions of summaries, and applying entailment module to judge how faithful these candidate summaries are with respect to the source text. We show that the proposed approach can select candidate summaries that improve ROUGE scores in all metrics evaluated.

More and more weakly structured, and irregular data sources are becoming available every day. The schema of these sources is useful for a number of tasks, such as query answering, exploration and summarization. However, although semantic web data might contain schema information, in many cases this is completely missing or partially defined. In this paper, we present a survey of the state of the art on schema information extraction approaches. We analyze and classify these approaches into three families: (1) approaches that exploit the implicit structure of the data, without assuming that some explicit statements on the schema are provided in the dataset; (2) approaches that use the explicit schema statements contained in the dataset to complement and enrich the schema, and (3) those that discover structural patterns contained in a dataset. We compare these studies in terms of their approach, advantages and limitations. Finally we discuss the problems that remain open.

Convolutional sequence to sequence (CNN seq2seq) models have met success in abstractive summarization. However, their outputs often contain repetitive word sequences and logical inconsistencies, limiting the practicality of their application. In this paper, we find the reasons behind the repetition problem in CNN-based abstractive summarization through observing the attention map between the summaries with repetition and their corresponding source documents and mitigate the repetition problem. We propose to reduce the repetition in summaries by attention filter mechanism (ATTF) and sentence-level backtracking decoder (SBD), which dynamically redistributes attention over the input sequence as the output sentences are generated. The ATTF can record previously attended locations in the source document directly and prevent the decoder from attending to these locations. The SBD prevents the decoder from generating similar sentences more than once via backtracking at test. The proposed model outperforms the baselines in terms of ROUGE score, repeatedness, and readability. The results show that this approach generates high-quality summaries with minimal repetition and makes the reading experience better.

Person search has long been treated as a crucial and challenging task to support deeper insight in personalized summarization and personality discovery. Traditional methods, e.g., person re-identification and face recognition techniques, which profile video characters based on visual information, are often limited by relatively fixed poses or small variation of viewpoints and suffer from more realistic scenes with high motion complexity (e.g., movies). At the same time, long videos such as movies often have logical story lines and are composed of continuously developmental plots. In this situation, different persons usually meet on a specific occasion, in which informative social cues are performed. We notice that these social cues could semantically profile their personality and benefit person search task in two aspects. First, persons with certain relationships usually co-occur in short intervals; in case one of them is easier to be identified, the social relation cues extracted from their co-occurrences could further benefit the identification for the harder ones. Second, social relations could reveal the association between certain scenes and characters (e.g., classmate relationship may only exist among students), which could narrow down candidates into certain persons with a specific relationship. In this way, high-level social relation cues could improve the effectiveness of person search. Along this line, in this article, we propose a social context-aware framework, which fuses visual and social contexts to profile persons in more semantic perspectives and better deal with person search task in complex scenarios. Specifically, we first segment videos into several independent scene units and abstract out social contexts within these scene units. Then, we construct inner-personal links through a graph formulation operation for each scene unit, in which both visual cues and relation cues are considered. Finally, we perform a relation-aware label propagation to identify characters’ occurrences, combining low-level semantic cues (i.e., visual cues) and high-level semantic cues (i.e., relation cues) to further enhance the accuracy. Experiments on real-world datasets validate that our solution outperforms several competitive baselines.

It is essential to provide a structured data feed to the computer to accomplish any task so that it can process flawlessly to generate the desired output within minimal computational time. Generally, computer programmers should provide a structured data feed to the computer program for its successful execution. The hardcopy document should be scanned to generate its corresponding computer-readable softcopy version of the file. This process also proves to be a budget-friendly approach to disengage human resources from the entire process of record maintenance. Due to this automation, the workload of existing manpower is reduced to a significant level. This concept may prove beneficial for the delivery of any type of services to the ultimate beneficiary (i.e., citizen) in a minimal time frame. The administration has to deal with various issues of citizens due to the pressure of a huge population who seek legal help to resolve their issues, thereby leading to the filing of large numbers of pending legal cases at several courts of the country. To assist the victims with prompt delivery of justice and legal professionals in reducing their workload, this paper proposed a machine learning based automated legal model to enhance the efficiency of the legal support system with an accuracy of 94%.

Recurrent Neural Networks (RNN) and their variants like Gated Recurrent Units (GRUs) have been the de-facto method in Natural Language Processing (NLP) for solving a range of NLP problems, including extractive text summarization. However, for certain sequential data with multiple temporal dependencies like the human text data, using a single RNN over the whole sequence might prove to be inadequate. Transformer models that use multiheaded attention have shown that human text contains multiple dependencies. Supporting networks like attention layers are needed to augment the RNNs to capture the numerous dependencies in text. In this work, we propose a novel combination of RNNs, called Parallel RNNs (PRNN), where small and narrow RNN units work on a sequence, in parallel and independent of each other, for the task of extractive text summarization. These PRNNs, without the need for any attention layers, capture various dependencies present in the sentence and document sequences. Our model achieved a 10% gain in ROUGE-2 score over the single RNN model on the popular CNN/Dailymail dataset. The boost in performance indicates that such an ensemble arrangement of RNNs improves the performance compared to the standard single RNNs, which alludes to the fact that constituent units of the PRNN learn various input sequence dependencies. Hence, the sequence is represented better using the combined representation from the constituent RNNs.

ICP-OES is a powerful, versatile, and advanced analytical technique with excellent detection properties. Due to its extraordinary features, it has been widely employed for the analysis of a wide variety of chemical elements in the past few years with great success. It offers the least detection time, lower detection limits, broader linear dynamic range, and greater matrix tolerability as well as negligible chemical interferences. Beyond this, it can handle multiple varieties of samples including aqueous, inorganic, organic liquids, and solids as well. It comprises complex instrumental makeup which enables it to detect up to 2 to 70 elements simultaneously with great accuracy. Recent reports evidenced that this hyphenated technique has been employed in several analytical determinations including food analysis, agricultural investigations, geological studies, drug/metabolite analysis, and environmental and forensic sciences. In the present compilation, a detailed description of the fundamental principles of ICP-OES has been provided along with its various sophisticated functional components. Also, the reported applications of this technique in different fields have been discussed highlighting the basic experimental setups and outcomes by presenting the data as tables. This summarization may be helpful to analysts to get insights into the working as well as performance characteristics of this technique.

Nowadays, fake news is widely spreading in various media, and this fake information is causing serious damage in many areas. Therefore, there is an increasing need to accurately detect fake news to prevent such damage. In this paper, we propose a novel method that uses graph and summarization techniques for fake news detection. Our proposed method represents the relationship of all sentences in a graph structure to accurately understand the context information of the document. Accordingly, the relationship between sentences in the graph is calculated as a score through the attention mechanism. Then, the summarization technique is used to reflect the sentence subject information in the graph update process. Our proposed method shows better performance than Karimi’s and BERT based models by approximately 10.34%p and 3.72%p, respectively.

Video summarization technologies aim to create a concise and complete synopsis by selecting the most informative parts of the video content. Several approaches have been developed over the last couple of decades and the current state of the art is represented by methods that rely on modern deep neural network architectures. This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization. After presenting the motivation behind the development of technologies for video summarization, we formulate the video summarization task and discuss the main characteristics of a typical deep-learning-based analysis pipeline. Then, we suggest a taxonomy of the existing algorithms and provide a systematic review of the relevant literature that shows the evolution of the deep-learning-based video summarization technologies and leads to suggestions for future developments. We then report on protocols for the objective evaluation of video summarization algorithms and we compare the performance of several deep-learning-based approaches. Based on the outcomes of these comparisons, as well as some documented considerations about the amount of annotated data and the suitability of evaluation protocols, we indicate potential future research directions., Accepted for publication at the Proceedings of the IEEE

Text summarization aims to reduce text by removing less useful information to obtain information quickly and precisely. In Indonesian abstractive text summarization, the research mostly focuses on multi-document summarization which methods will not work optimally in single-document summarization. As the public summarization datasets and works in English are focusing on single-document summarization, this study emphasized on Indonesian single-document summarization. Abstractive text summarization studies in English frequently use Bidirectional Encoder Representations from Transformers (BERT), and since Indonesian BERT checkpoint is available, it was employed in this study. This study investigated the use of Indonesian BERT in abstractive text summarization on the IndoSum dataset using the BERTSum model. The investigation proceeded by using various combinations of model encoders, model embedding sizes, and model decoders. Evaluation results showed that models with more embedding size and used Generative Pre-Training (GPT)-like decoder could improve the Recall-Oriented Understudy for Gisting Evaluation (ROUGE) score and BERTScore of the model results.

Nowadays, opinion texts are quickly published on websites and social networks by various users in the form of short texts and also in high volumes and various fields. Because these texts reflect the opinions of many users, their processing and analysis, such as clustering, can be very useful in a variety of applications including politics, industry, commerce, and economics. High dimensions of the text representation decrease efficiency of clustering, and an effective solution for this challenge is reducing dimensions of texts. Manifold learning is a powerful tool for nonlinear dimension reduction of high-dimensional data. Therefore, in this paper, for increasing efficiency of opinion texts clustering, by manifold learning, dimensions of the represented opinion texts are reduced based on sentiment and semantics, and their intrinsic dimensions are extracted. Then, the clustering algorithm is applied to dimension-reduced opinion texts. The proposed approach helps us to cluster opinion texts with simultaneous consideration of sentiment and semantics, which has received very little attention in the previous works. This type of clustering helps users of opinion texts to obtain more useful information from texts and also provides more accurate summaries in applications, such as the summarization of opinion texts. Experimental results on three datasets show better performance of the proposed approach on opinion texts in terms of important measures for evaluating clustering efficiency. An improvement of about 9% is observed in terms of accuracy on the third dataset and clustering based on sentiment and semantics.

Neural sequence-to-sequence models are the state-of-the-art approach used in abstractive summarization of textual documents, useful for producing condensed versions of source text narratives without being restricted to using only words from the original text. Despite the advances in abstractive summarization, custom generation of summaries (e.g., towards a user’s preference) remains unexplored. In this article, we present CATS, an abstractive neural summarization model that summarizes content in a sequence-to-sequence fashion while also introducing a new mechanism to control the underlying latent topic distribution of the produced summaries. We empirically illustrate the efficacy of our model in producing customized summaries and present findings that facilitate the design of such systems. We use the well-known CNN/DailyMail dataset to evaluate our model. Furthermore, we present a transfer-learning method and demonstrate the effectiveness of our approach in a low resource setting, i.e., abstractive summarization of meetings minutes, where combining the main available meetings’ transcripts datasets, AMI and International Computer Science Institute(ICSI) , results in merely a few hundred training documents.

Data users are generally interested in two types of aggregated information: summarization of the selected attribute(s) for all considered entities and retrieval and evaluation of entities by the requirements posed on the relevant attributes. Less statistically literate users (e.g., domain experts) and the business intelligence strategic dashboards can benefit from linguistic summarization, i.e. a summary like most customers are middle-aged can be understood immediately. Evaluation of the mandatory and optional requirements of the structure P1 and most of the other posed predicates should be satisfied beneficial for analytical business intelligence dashboards and search engines in general. This work formalizes the integration of the aforementioned quantified summaries and quantified evaluation into the concept of database queries to empower their flexibility by, e.g., the nested quantified query conditions on hierarchical data structures. Later in our work, we adapted our research into practical application. We created a software environment for evaluating data based on a dataset retrieved from The Statistical Office of the Slovak republic. These datasets are aimed mainly on landscape characteristics like altitude, area sizes of towns and villages, and similar parameters. Based on user's preferences, our system recommends the most suitable place for holidays to spend on.

The traditional frequency based approach to creating multi-document extractive summary ranks sentences based on scores computed by summing up TF*IDF weights of words contained in the sentences. In this approach, TF or term frequency is calculated based on how frequently a term (word) occurs in the input and TF calculated in this way does not take into account the semantic relations among terms. In this paper, we propose methods that exploits semantic term relations for improving sentence ranking and redundancy removal steps of a summarization system. Our proposed summarization system has been tested on DUC 2003 and DUC 2004 benchmark multi-document summarization datasets. The experimental results reveal that performance of our multi-document text summarizer is significantly improved when the distributional term similarity measure is used for finding semantic term relations. Our multi-document text summarizer also outperforms some well known summarization baselines to which it is compared.