Search

Short text clustering has far-reaching effects on semantic analysis, showing its importance for multiple applications such as corpus summarization and information retrieval. However, it inevitably encounters the severe sparsity of short text representations, making the previous clustering approaches still far from satisfactory. In this paper, we present a novel attentive representation learning model for shot text clustering, wherein cluster-level attention is proposed to capture the correlations between text representations and cluster representations. Relying on this, the representation learning and clustering for short texts are seamlessly integrated into a unified model. To further ensure robust model training for short texts, we apply adversarial training to the unsupervised clustering setting, by injecting perturbations into the cluster representations. The model parameters and perturbations are optimized alternately through a minimax game. Extensive experiments on four real-world short text datasets demonstrate the superiority of the proposed model over several strong competitors, verifying that robust adversarial training yields substantial performance gains., Comment: 14pages, to appear in IEEE TKDE

Extracting relevant information from a large amount of data is a challenging task. Automatic text summarization is a potential solution for obtaining this information. In this paper, a nature inspired swarm intelligence-based algorithm viz. firefly algorithm for multi-document text summarization is proposed. A new fitness function consisting of three features viz. topic relation factor, cohesion factor and readability factor is utilized. The experiments are performed on datasets from Document Understanding Conference i.e. DUC-2002, DUC-2003 and DUC-2004. The performance of the algorithm has been evaluated using ROUGE score. The performance of the proposed algorithm is compared with some other nature inspired ones such as particle swarm optimization (PSO) and genetic algorithm (GA). The performance of the proposed algorithm outperforms the other adopted ones.

Update summarization is a relatively recent summarization task concerned with creating a short summary from news articles, assuming the user has already read a number of previous articles. It is useful for users who want to know the latest information about a specific topic. The availability of systems that provide update summaries saves user time and effort. Unfortunately, such resources are lacking for the Arabic language. This paper aims to provide an update summarization system that generates an update summary containing the latest information requested by a user from multiple documents. This paper provides an Arabic query-based Update-Summarization System (AUSS). We use a graph-based ranking model to represent the similarity, through a combination of lexical and semantic relations between words. Our experiments show that AUSS achieves promising results, achieving a best F-Measure of 0.5405. To test AUSS, we created a new corpus especially for Arabic Update Summaries, which we call (AUS-DB). AUS-DB contains 183 articles and their corresponding reference summaries.

The availability of legal judgment documents in digital form offers numerous opportunities for information extraction and application. Automatic summarization of these legal texts is a crucial and a challenging task due to the unusual structure and high complexity of these documents. Previous approaches in this direction have relied on huge labelled datasets, using hand engineered features, leveraging on domain knowledge and focussed their attention on a narrow sub-domain for increased effectiveness. In this paper, we propose simple generic techniques using neural network for the summarization task for Indian legal judgment documents. We explore two neural network architectures for this task utilizing the word and sentence embeddings for capturing the semantics. The main advantage of the proposed approaches is that they do not rely on hand crafted features, or domain specific knowledge, nor is their application restricted to a particular sub-domain thus making them suitable to be extended to other domains as well. We tackle the problem of unavailability of labelled data for the task by assigning classes/scores to sentences in the training set, based on their match with reference summary produced by humans. The experimental evaluations establish the effectiveness of our proposed approaches as compared with other baselines.

Document clustering has recently been paid great attention in retrieval, navigation, and summarization of huge volumes of documents. With a better document clustering approach, computers can organize a document corpus automatically to a meaningful cluster for enabling efficient navigation, and browsing of the corpus. Document navigation and browsing is a valuable complement to the deficiencies of information retrieval technologies. This paper introduces Modsup-based frequent itemset and Rider Optimization-based Moth Search Algorithm (Rn-MSA) for clustering the documents. At first, the input documents are given to the pre-processing step, and then, the extraction is carried out based on TF-IDF and Wordnet features. Once the extraction is done, the feature selection is carried out based on frequent itemset for the establishment of feature knowledge. At last, the document clustering is done using the proposed Rn-MSA, which is designed by combining Rider Optimization Algorithm (ROA), and the Moth Search Algorithm (MSA). The performance of the document clustering based on proposed Modsup + Rn-MSA is evaluated in terms of precision, recall, F-Measure, and accuracy. The developed document clustering method achieves the maximal precision of 95.90%, maximal recall of 96.41%, maximal F-Measure of 96.41%, and the maximal accuracy of 95.12% that indicates its superiority.

Text summarization automatically produces a summary containing important sentences and includes all relevant important information from the original document. One of the main approaches, when viewed from the summary results, are extractive and abstractive. An extractive summary is heading towards maturity and now research has shifted towards abstractive summation and real-time summarization. Although there have been so many achievements in the acquisition of datasets, methods, and techniques published, there are not many papers that can provide a broad picture of the current state of research in this field. This paper provides a broad and systematic review of research in the field of text summarization published from 2008 to 2019. There are 85 journal and conference publications which are the results of the extraction of selected studies for identification and analysis to describe research topics/trends, datasets, preprocessing, features, techniques, methods, evaluations, and problems in this field of research. The results of the analysis provide an in-depth explanation of the topics/trends that are the focus of their research in the field of text summarization; provide references to public datasets, preprocessing and features that have been used; describes the techniques and methods that are often used by researchers as a comparison and means for developing methods. At the end of this paper, several recommendations for opportunities and challenges related to text summarization research are mentioned.

The scientific research process generally starts with the examination of the state of the art, which may involve a vast number of publications. Automatically summarizing scientific articles would help researchers in their investigation by speeding up the research process. The automatic summarization of scientific articles differs from the summarization of generic texts due to their specific structure and inclusion of citation sentences. Most of the valuable information in scientific articles is presented in tables, figures, and algorithm pseudocode. These elements, however, do not usually appear in a generic text. Therefore, several approaches that consider the particularity of a scientific article structure were proposed to enhance the quality of the generated summary, resulting in ad hoc automatic summarizers. This paper provides a comprehensive study of the state of the art in this field and discusses some future research directions. It particularly presents a review of approaches developed during the last decade, the corpora used, and their evaluation methods. It also discusses their limitations and points out some open problems. The conclusions of this study highlight the prevalence of extractive techniques for the automatic summarization of single monolingual articles using a combination of statistical, natural language processing, and machine learning techniques. The absence of benchmark corpora and gold standard summaries for scientific articles remains the main issue for this task.

Extraction of knowledge from soccer videos has enormous applications like context-based advertisement, content-based video retrieval, match summarization, and highlight extraction. Overlapping soccer actions and uncontrolled video capturing conditions make it challenging to detect action accurately. For overcoming these problems, Convolutional Neural Network and Recurrent Neural Network are used jointly to classify different lengths of soccer actions. Initially, transfer learning from pre-trained VGG network extracts characteristic spatial features. Afterwards, Gated Recurrent Unit deals with temporal dependency and solves the vanishing gradient problem. Finally, softmax layer assigns decimal probabilities to each class. Experimental results demystify the significance of the proposed architecture.

Ever since the first introduction of Artificial Intelligence into the field of hydrology, it has further generated immense interest in researching aspects for further improvements to hydrology. This can be seen in the rising number of related works published. This culminated further with the combination of pioneering optimization techniques. Who would have thought that the birds and the bees can offer advances in the mathematical sciences and so have the ants too? The ingenuity of humans is spelled out in the algorithms that mimic many natural activities, like pack hunting by the wolves! This review paper serves to broadcast more of the intriguing interest in newfound procedures in optimal forecasting. Reservoirs are the main and most efficient water storage facilities for managing uneven water distribution. However, due to the major global climate changes which affect rainfall trend and weather, it has been a necessity to find an alternative solution for effective conventional water balance. A multifunctional reservoir operation appears to require the operator to make wise decisions to achieve an optimal reservoir operation. One of the most important aspects of all this is the forecasting of streamflows. For this, Artificial Intelligence (AI) seems to be the best alternative solution; as in the past three decades, there has been a drastic increase in building and developing AI models for forecasting and modelling unstable patterns in various hydrological fields. Nevertheless, AI models are also required to be optimized in tandem to achieve the best result, leading thus to the desirous forming of hybrid models between a standalone AI model and optimization techniques. This comprehensive study categorizes machine learning into three main categories, together with the optimization techniques, and will next explore the various AI model used for different hydrology fields along with the most common optimization techniques. Summarization of findings under every section is provided. Some advantages and disadvantages found through literature reviews are summarized for ease of reference. Finally, future recommendations and overall conclusions drawn from the results of researchers are included. This current review focuses on papers from high-impact factor publications based on 10 years starting from (2009 to 2020).

Code summarization (aka comment generation) provides a high-level natural language description of the function performed by code, which can benefit the software maintenance, code categorization and retrieval. To the best of our knowledge, the state-of-the-art approaches follow an encoder-decoder framework which encodes source code into a hidden space and later decodes it into a natural language space. Such approaches suffer from the following drawbacks: (a) they are mainly input by representing code as a sequence of tokens while ignoring code hierarchy; (b) most of the encoders only input simple features (e.g., tokens) while ignoring the features that can help capture the correlations between comments and code; (c) the decoders are typically trained to predict subsequent words by maximizing the likelihood of subsequent ground truth words, while in real world, they are excepted to generate the entire word sequence from scratch. As a result, such drawbacks lead to inferior and inconsistent comment generation accuracy. To address the above limitations, this paper presents a new code summarization approach using hierarchical attention network by incorporating multiple code features, including type-augmented abstract syntax trees and program control flows. Such features, along with plain code sequences, are injected into a deep reinforcement learning (DRL) framework (e.g., actor-critic network) for comment generation. Our approach assigns weights (pays “attention”) to tokens and statements when constructing the code representation to reflect the hierarchical code structure under different contexts regarding code features (e.g., control flows and abstract syntax trees). Our reinforcement learning mechanism further strengthens the prediction results through the actor network and the critic network, where the actor network provides the confidence of predicting subsequent words based on the current state, and the critic network computes the reward values of all the possible extensions of the current state to provide global guidance for explorations. Eventually, we employ an advantage reward to train both networks and conduct a set of experiments on a real-world dataset. The experimental results demonstrate that our approach outperforms the baselines by around 22% to 45% in BLEU-1 and outperforms the state-of-the-art approaches by around 5% to 60% in terms of S-BLEU and C-BLEU.

Building a visual overview of temporal event sequences with an optimal level-of-detail (i.e. simplified but informative) is an ongoing challenge - expecting the user to zoom into every important aspect of the overview can lead to missing insights. We propose a technique to build a multilevel overview of event sequences, whose granularity can be transformed across sequence clusters (vertical level-of-detail) or longitudinally (horizontal level-of-detail), using hierarchical aggregation and a novel cluster data representation Align-Score-Simplify. By default, the overview shows an optimal number of sequence clusters obtained through the average silhouette width metric - then users are able to explore alternative optimal sequence clusterings. The vertical level-of-detail of the overview changes along with the number of clusters, whilst the horizontal level-of-detail refers to the level of summarization applied to each cluster representation. The proposed technique has been implemented into a visualization system called Sequence Cluster Explorer (Sequen-C) that allows multilevel and detail-on-demand exploration through three coordinated views, and the inspection of data attributes at cluster, unique sequence, and individual sequence level. We present two case studies using real-world datasets in the healthcare domain: CUREd and MIMIC-III; which demonstrate how the technique can aid users to obtain a summary of common and deviating pathways, and explore data attributes for selected patterns., Comment: This is the author's version of the article to be published in IEEE Transactions on Visualization and Computer Graphics

Due to recent advances in the film industry, the production of movies has grown exponentially, which has led to challenges in what is referred to as discoverability: given the overwhelming number of choices, choosing which film to watch has become a tedious task for audiences. Movie summarization (MS) could help, as it presents the central theme of the movie in a compact format and makes browsing more efficient for the audience. In this paper, we present an automatic MS framework coined as ‘QuickLook’, which identifies the leading characters and fuses multiple cues extracted from a movie. Firstly, the movie data is preprocessed for its division into scenes, followed by shot segmentation. Secondly, the leading characters in each segmented scene are determined. Next, four visual cues that capture the film's scenic beauty, memorability, informativeness and emotional resonance are extracted from shots containing the leading characters. These extracted features are then intelligently fused based on the assignment of different weights; shots with a fusion score above a certain threshold are selected for the final summary. The proposed MS framework is assessed by comparison with official trailers from ten Hollywood movies, providing a novel baseline for future fair comparison in the MS literature. The proposed framework is shown to outperform other state-of-the-art MS methods in terms of enjoyability and informativeness.

Aggregation functions, which are at the heart of a number of information fusion processes, allow summarization of multiple inputs into a single representative value. Extended aggregation functions are defined such that the input data can be of varying cardinality, with the implication that there is some consistency across the methods of calculation. This article formalizes an approach to extended aggregation such that contributions of repeated inputs or regions of high density are limited in their ability to influence the final value. We establish important definitions and properties, in particular around whether such functions will be monotone or directionally monotone. We then propose a powerful construction method for extended pre-aggregation functions. Illustrative examples are provided throughout.

It is essential to provide a structured data feed to the computer to accomplish any task so that it can process flawlessly to generate the desired output within minimal computational time. Generally, computer programmers should provide a structured data feed to the computer program for its successful execution. The hardcopy document should be scanned to generate its corresponding computer-readable softcopy version of the file. This process also proves to be a budget-friendly approach to disengage human resources from the entire process of record maintenance. Due to this automation, the workload of existing manpower is reduced to a significant level. This concept may prove beneficial for the delivery of any type of services to the ultimate beneficiary (i.e., citizen) in a minimal time frame. The administration has to deal with various issues of citizens due to the pressure of a huge population who seek legal help to resolve their issues, thereby leading to the filing of large numbers of pending legal cases at several courts of the country. To assist the victims with prompt delivery of justice and legal professionals in reducing their workload, this paper proposed a machine learning based automated legal model to enhance the efficiency of the legal support system with an accuracy of 94%.

Video summarization technologies aim to create a concise and complete synopsis by selecting the most informative parts of the video content. Several approaches have been developed over the last couple of decades and the current state of the art is represented by methods that rely on modern deep neural network architectures. This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization. After presenting the motivation behind the development of technologies for video summarization, we formulate the video summarization task and discuss the main characteristics of a typical deep-learning-based analysis pipeline. Then, we suggest a taxonomy of the existing algorithms and provide a systematic review of the relevant literature that shows the evolution of the deep-learning-based video summarization technologies and leads to suggestions for future developments. We then report on protocols for the objective evaluation of video summarization algorithms and we compare the performance of several deep-learning-based approaches. Based on the outcomes of these comparisons, as well as some documented considerations about the amount of annotated data and the suitability of evaluation protocols, we indicate potential future research directions., Accepted for publication at the Proceedings of the IEEE

Neural sequence-to-sequence models are the state-of-the-art approach used in abstractive summarization of textual documents, useful for producing condensed versions of source text narratives without being restricted to using only words from the original text. Despite the advances in abstractive summarization, custom generation of summaries (e.g., towards a user’s preference) remains unexplored. In this article, we present CATS, an abstractive neural summarization model that summarizes content in a sequence-to-sequence fashion while also introducing a new mechanism to control the underlying latent topic distribution of the produced summaries. We empirically illustrate the efficacy of our model in producing customized summaries and present findings that facilitate the design of such systems. We use the well-known CNN/DailyMail dataset to evaluate our model. Furthermore, we present a transfer-learning method and demonstrate the effectiveness of our approach in a low resource setting, i.e., abstractive summarization of meetings minutes, where combining the main available meetings’ transcripts datasets, AMI and International Computer Science Institute(ICSI) , results in merely a few hundred training documents.

Multi-modal abstractive summarization for videos is an emerging task, aiming to integrate multi-modal and multi-source inputs (video, audio transcript) into a compressed textual summary. Although recent multi-encoder-decoder models on this task have shown promising performance, they did not explicitly model interactions of multi-source inputs. While some strategies like co-attention are utilized for modeling this interaction, considering ultra-long sequences and additional decoder in this task, the coupling of multi-modal data from multi-encoders and decoder needs complicated structure and additional parameters. In this paper, we propose a concise Decoder-only Multi-modal Transformer (D-MmT) based on the above observations. Specifically, we cut the encoder structure, and introduce an in-out shared multi-modal decoder to make the multi-source and target fully interact and couple in the shared feature space, reducing the model parameter redundancy. Also, we design a concise cascaded cross-modal interaction (CXMI) module in the multi-modal decoder that generates joint fusion representations and spontaneously establishes a fine-grained intra- and inter- association between multi-modalities. In addition, to make full use of the ultra-long sequence information, we introduce a joint in-out loss to make the input transcript also participate in backpropagation to enhance the contextual feature representation. The experimental results on the How2 dataset show that the proposed model outperforms the current state-of-the-art approach with fewer model parameters. Further analysis and visualization show the effectiveness of our proposed framework.

Application Programming Interfaces (APIs) have been widely discussed on social-technical platforms (e.g., Stack Overflow). Extracting API mentions from such informal software texts is the prerequisite for API-centric search and summarization of programming knowledge. Machine learning based API extraction has demonstrated superior performance than rule-based methods in informal software texts that lack consistent writing forms and annotations. However, machine learning based methods have a significant overhead in preparing training data and effective features. In this paper, we propose a multi-layer neural network based architecture for API extraction. Our architecture automatically learns character-, word- and sentence-level features from the input texts, thus removing the need for manual feature engineering and the dependence on advanced features (e.g., API gazetteers) beyond the input texts. We also propose to adopt transfer learning to adapt a source-library-trained model to a target-library, thus reducing the overhead of manual training-data labeling when the software text of multiple programming languages and libraries need to be processed. We conduct extensive experiments with six libraries of four programming languages which support diverse functionalities and have different API-naming and API-mention characteristics. Our experiments investigate the performance of our neural architecture for API extraction in informal software texts, the importance of different features, the effectiveness of transfer learning. Our results confirm not only the superior performance of our neural architecture than existing machine learning based methods for API extraction in informal software texts, but also the easy-to-deploy characteristic of our neural architecture.

Automatic text summarization extracts important information from texts and presents the information in the form of a summary. Abstractive summarization approaches progressed significantly by switching to deep neural networks, but results are not yet satisfactory, especially for languages where large training sets do not exist. In several natural language processing tasks, a cross-lingual model transfer is successfully applied in less-resource languages. For summarization, the cross-lingual model transfer was not attempted due to a non-reusable decoder side of neural models that cannot correct target language generation. In our work, we use a pre-trained English summarization model based on deep neural networks and sequence-to-sequence architecture to summarize Slovene news articles. We address the problem of inadequate decoder by using an additional language model for the evaluation of the generated text in target language. We test several cross-lingual summarization models with different amounts of target data for fine-tuning. We assess the models with automatic evaluation measures and conduct a small-scale human evaluation. Automatic evaluation shows that the summaries of our best cross-lingual model are useful and of quality similar to the model trained only in the target language. Human evaluation shows that our best model generates summaries with high accuracy and acceptable readability. However, similar to other abstractive models, our models are not perfect and may occasionally produce misleading or absurd content.

The automatic correction of grammar and spelling errors is important for students, second language learners, and some Natural Language Processing (NLP) tasks such as part of speech and text summarization. Recently, Neural Machine Translation (NMT) has been an out-performing and well-established model in the task of Grammar Error Correction (GEC). Arabic GEC is still growing because of some challenges, such as scarcity of training sets and the complexity of Arabic language. To overcome these issues, we introduced an unsupervised method to generate large-scale synthetic training data based on confusion function to increase the amount of training set. Furthermore, we introduced a supervised NMT model for AGEC called SCUT AGEC. SCUT AGEC is a convolutional sequence-to-sequence model consisting of nine encoder-decoder layers with attention mechanism. We applied fine-tuning to improve the performance and get more efficient results. Convolutional Neural Networks (CNN) gives our model ability to joint feature extraction and classification in one task and we proved that it is an efficient way to capture features of the local context. Moreover, it is easy to obtain long-term dependencies because of convolutional layers staking. Our proposed model becomes the first supervised AGEC system based on the convolutional sequence-to-sequence learning to outperforms the current state-of-the-art neural AGEC models.

Objective Biomedical text summarization helps biomedical information seekers avoid information overload by reducing the length of a document while preserving the contents’ essence. Our systematic review investigates the most recent biomedical text summarization researches on biomedical literature and electronic health records by analyzing their techniques, areas of application, and evaluation methods. We identify gaps and propose potential directions for future research. Materials and Methods This review followed the PRISMA methodology and replicated the approaches adopted by the previous systematic review published on the same topic. We searched 4 databases (PubMed, ACM Digital Library, Scopus, and Web of Science) from January 1, 2013 to April 8, 2021. Two reviewers independently screened title, abstract, and full-text for all retrieved articles. The conflicts were resolved by the third reviewer. The data extraction of the included articles was in 5 dimensions: input, purpose, output, method, and evaluation. Results Fifty-eight out of 7235 retrieved articles met the inclusion criteria. Thirty-nine systems used single-document biomedical research literature as their input, 17 systems were explicitly designed for clinical support, 47 systems generated extractive summaries, and 53 systems adopted hybrid methods combining computational linguistics, machine learning, and statistical approaches. As for the assessment, 51 studies conducted an intrinsic evaluation using predefined metrics. Discussion and Conclusion This study found that current biomedical text summarization systems have achieved good performance using hybrid methods. Studies on electronic health records summarization have been increasing compared to a previous survey. However, the majority of the works still focus on summarizing literature.

The Seq2Seq abstract summarization model based on long short-term memory (LSTM) is very effective for short text summarization. However, LSTM is limited by long-term dependencies, which can potentially result in salient information loss when long text is processed by the Seq2Seq model based on LSTM. To overcome the long-term dependence limitation, an encoder-decoder model based on the dynamic residual network is proposed in this work. The model can dynamically select an optimal state from the state history to establish a connection with the current state to improve the LSTM long sequence dependencies according to the current decoding environment. Because the dynamic residual connections will result in long-term connection-dependent words, a new method based on reinforcement learning is proposed to simulate the dependence between words, which is then implemented into the training process of the model. This model is verified using the CNN/Daily Mail and New York Times datasets, and the experimental results show that the proposed model achieves significant improvements in capturing long-term dependencies compared with the traditional LSTM-based Seq2Seq abstractive summarization model.

Nowadays, rich quantity of information is offered on the Net which makes it hard for the clients to detect necessary information. Programmed techniques are desirable to effectively filter and search useful data from the Net. The purpose of purported text summarization is to get satisfied content handling with information variety. The main factor of document summarization is to extract benefit feature. In this paper, we extract word feature in three group called important words. Also, we extract sentence feature depending on the extracted words. With increasing knowledge on the Internet, it turns out to be an extremely time-consuming, exhausting, and boring mission to read the whole content and papers and get the relevant information on precise topics

Lifelog analytics is an emerging research area with technologies embracing the latest advances in machine learning, wearable computing, and data analytics. However, state-of-the-art technologies are still inadequate to distill voluminous multimodal lifelog data into high quality insights. In this article, we propose a novel semantic relevance mapping ( SRM ) method to tackle the problem of lifelog information access. We formulate lifelog image retrieval as a series of mapping processes where a semantic gap exists for relating basic semantic attributes with high-level query topics. The SRM serves both as a formalism to construct a trainable model to bridge the semantic gap and an algorithm to implement the training process on real-world lifelog data. Based on the SRM, we propose a computational framework of lifelog analytics to support various applications of lifelog information access, such as image retrieval, summarization, and insight visualization. Systematic evaluations are performed on three challenging benchmarking tasks to show the effectiveness of our method.

In this paper, we propose a novel query generation task we refer to as the Strong Natural Language Query (SNLQ) problem. The key idea we explore is how to best learn document summarization and ranker effectiveness jointly in order to generate human-readable queries which capture the information need conveyed by a document, and that can also be used for refinding tasks and query rewriting. Our problem is closely related to two well-known retrieval problems—known-item finding and strong query generation—with the additional objective of maximizing query informativeness. In order to achieve this goal, we combine state-of-the-art abstractive summarization techniques and reinforcement learning. We have empirically compared our new approaches with several closely related baselines using the MS-MARCO data collection, and show that the approach is capable of achieving substantially better trade-off between effectiveness and human-readability than have been reported previously.

This article is an attempt to comprehend the difficulties and propose approaches to eliminate them when analyzing legal documents in the framework of economic and interdisciplinary research. The utmost goal is to seek incorporating advances in computational linguistics and natural language analysis into the discourse of the digital economy in order to develop methods involved in decision-making and strategy development based on the analysis of textual information. In conditions when the amount of information is too large, is constantly updated and / or the area of study is new, the most expedient at the first stage is to obtain the general structure of the entire collection of documents, some kind of semantic compression of information. The practical part contains the development of an approach for the analysis of regulations governing food and nutrition issues, in particular, related to the prevention of the development of iron deficiency anemia (IDA). The approach includes the extraction of key information of voluminous texts (keywords and key sentences) based on the TextRank graph algorithm. An important link contributing to cognition is also the visualization of semantic relationships between words within documents. In our opinion, it is the combination of semantic compression and visualization of information as a “close-up” of text documents, as well as the possibility of further detailing by linear reading and analysis, which are the most relevant approach in conditions of information overload and attention deficit. The active introduction of text analytics methods for systems that are not involved in attention markets, which lag significantly behind in the convenience of extracting meaningful information, is especially important. Approaches to improve the understanding of large volumes of regulations will be of significant value to researchers in economic, legal or multidisciplinary research.

Analyzing students’ emotions from classroom videos can help both teachers and parents quickly know the engagement of students in class. The availability of high-definition cameras creates opportunities to record class scenes. However, watching videos is time-consuming, and it is challenging to gain a quick overview of the emotion distribution and find abnormal emotions. In this article, we propose EmotionCues , a visual analytics system to easily analyze classroom videos from the perspective of emotion summary and detailed analysis, which integrates emotion recognition algorithms with visualizations. It consists of three coordinated views: a summary view depicting the overall emotions and their dynamic evolution, a character view presenting the detailed emotion status of an individual, and a video view enhancing the video analysis with further details. Considering the possible inaccuracy of emotion recognition, we also explore several factors affecting the emotion analysis, such as face size and occlusion. They provide hints for inferring the possible inaccuracy and the corresponding reasons. Two use cases and interviews with end users and domain experts are conducted to show that the proposed system could be useful and effective for analyzing emotions in the classroom videos.

The exponential growth of online textual data triggered the crucial need for an effective and powerful tool that automatically provides the desired content in a summarized form while preserving core information. In this paper, we propose an automatic, generic, and extractive Arabic single document summarizing method aiming at producing a sufficiently informative summary. The proposed extractive method evaluates each sentence based on a combination of statistical and semantic features in which a novel formulation is used taking into account sentence importance, coverage and diversity. Further, two summarizing techniques including score-based and supervised machine learning were employed to produce the summary and then assist leveraging the designed features. We demonstrate the effectiveness of the proposed method through a set of experiments under EASC corpus using ROUGE measure. Compared to some existing related work, the experimental evaluation shows the strength of the proposed method in terms of precision, recall, and F-score performance metrics.

Product reviews are of great commercial value for online shopping market. The identification of customer opinions from product reviews is helpful to improve the marketing decisions of customers, sellers and producers. This paper proposes a novel framework for summarizing customer opinions from product reviews. Firstly, our framework identifies grammatically and semantically meaningful phrases which contain product attributes and their corresponding opinions from original product reviews by using grammar rules and the latent Dirichlet allocation (LDA) model. Secondly, our framework generates readable and simple summaries from the identified phrases automatically by using the deep neural network. The summaries provide users the valuable opinions on product attributes. Moreover, our framework provides an interactive mode for users to choose product attributes which they are interested for generate personalized summaries to help users focus on the most concerned opinions. Experimental results on six datasets demonstrate effectiveness of our framework.

With the increasing amounts of electronic health data being constantly generated in medical examinations and by sensors and mobile applications, data visualization methods can assist medical professionals and researchers in exploring and making sense of the data. Two important challenges faced by data visualization are large data volume and protection of sensitive data. In this paper, we propose a graph-based method that allows the exploration of a patient dataset, while also naturally allowing the summarization of large amounts of data, making it applicable to large datasets and sensitive data. A graph is constructed from the raw data, encoding local similarities among patients, and is visualized on the screen, producing a visual map of the patient distribution. Multidimensional glyphs are put in place of the nodes, revealing the properties that characterize each graph area. The graph construction method is extended to an incremental scheme, allowing federated graph formation. The proposed method is demonstrated in three use cases, regarding frailty in older adults, Sjogren’s Syndrome patients, and a large-size diabetes dataset.

Purpose Query-based summarization approaches might not be able to provide summaries compatible with the user’s information need, as they mostly rely on a limited source of information, usually represented as a single query by the user. This issue becomes even more challenging when dealing with scientific documents, as they contain more specific subject-related terms, while the user may not be able to express his/her specific information need in a query with limited terms. This study aims to propose an interactive multi-document text summarization approach that generates an eligible summary that is more compatible with the user’s information need. This approach allows the user to interactively specify the composition of a multi-document summary. Design/methodology/approach This approach exploits the user’s opinion in two stages. The initial query is refined by user-selected keywords/keyphrases and complete sentences extracted from the set of retrieved documents. It is followed by a novel method for sentence expansion using the genetic algorithm, and ranking the final set of sentences using the maximal marginal relevance method. Basically, for implementation, the Web of Science data set in the artificial intelligence (AI) category is considered. Findings The proposed approach receives feedback from the user in terms of favorable keywords and sentences. The feedback eventually improves the summary as the end. To assess the performance of the proposed system, this paper has asked 45 users who were graduate students in the field of AI to fill out a questionnaire. The quality of the final summary has been also evaluated from the user’s perspective and information redundancy. It has been investigated that the proposed approach leads to higher degrees of user satisfaction compared to the ones with no or only one step of the interaction. Originality/value The interactive summarization approach goes beyond the initial user’s query, while it includes the user’s preferred keywords/keyphrases and sentences through a systematic interaction. With respect to these interactions, the system gives the user a more clear idea of the information he/she is looking for and consequently adjusting the final result to the ultimate information need. Such interaction allows the summarization system to achieve a comprehensive understanding of the user’s information needs while expanding context-based knowledge and guiding the user toward his/her information journey.

Developing an abstractive text summarization (ATS) system that is capable of generating concise, appropriate, and plausible summaries for the source documents is a long-term goal of artificial intelligence (AI). Recent advances in ATS are overwhelmingly contributed by deep learning techniques, which have taken the state-of-the-art of ATS to a new level. Despite the significant success of previous methods, generating high-quality and human-like abstractive summaries remains a challenge in practice. The human reading cognition, which is essential for reading comprehension and logical thinking, is still relatively new territory and underexplored in deep neural networks. In this article, we propose a novel Hierarchical Human-like deep neural network for ATS (HH-ATS), inspired by the process of how humans comprehend an article and write the corresponding summary. Specifically, HH-ATS is composed of three primary components (i.e., a knowledge-aware hierarchical attention module, a multitask learning module, and a dual discriminator generative adversarial network), which mimic the three stages of human reading cognition (i.e., rough reading, active reading, and postediting). Experimental results on two benchmark data sets (CNN/Daily Mail and Gigaword) demonstrate that HH-ATS consistently and substantially outperforms the compared methods.

First-person video summarization has emerged as an important research problem for computer vision and multimedia communities. In this paper, we show how different graph representations can be developed for accurately summarizing first-person (egocentric) videos in a computationally efficient manner. Each frame in a video is first represented as a weighted graph. A shot boundary detection method using graph based mutual information is developed. We next construct a weighted graph for each shot. A representative frame from each shot is selected using a graph centrality measure. A new way of characterizing egocentric video frames using a graph based center-surround model is shown next. Here, each representative frame is modeled as a union of a center region (graph) and a surround region (graph). By exploiting spectral measures of dissimilarity between the two (center and surround) graphs, optimal center and surround regions are determined. Optimal regions for all frames within a shot are kept the same as that of the representative frame. Center-surround differences in entropy and optical flow values along with PHOG (Pyramidal HOG) features are extracted from each frame. All frames in a video are finally represented by another weighted graph, termed as a Video Similarity Graph (VSG). The frames are clustered by applying a Minimum Spanning Tree (MST) based approach with a new measure for inadmissible edges. Frames closest to the centroid of each cluster are captured to build the summary. Experimental evaluation on two benchmark datasets indicate the advantage of the proposed formulation.

There is an exponential growth of text data over the internet, and it is expected to gain significant growth and attention in the coming years. Extracting meaningful insights from text data is crucially important as it offers value-added solutions to business organizations and end-users. Automatic text summarization (ATS) automates text summarization by reducing the initial size of the text without the loss of key information elements. In this article, we propose a novel text summarization algorithm for documents using Deep Learning Modifier Neural Network (DLMNN) classifier. It generates an informative summary of the documents based on the entropy values. The proposed DLMNN framework comprises six phases. In the initial phase, the input document is pre-processed. Subsequently, the features are extracted using pre-processed data. Next, the most appropriate features are selected using the improved fruit fly optimization algorithm (IFFOA). The entropy value for every chosen feature is computed. These values are then classified into two classes, (a) highest entropy values and (b) lowest entropy values. Finally, the class that holds the highest entropy values is chosen, representing the informative sentences that form the last summary. The results observed from the experiment indicate that the DLMNN classifier gives 81.56, 91.21, and 83.53 of sensitivity, accuracy, specificity, precision, and f-measure. Whereas the existing schemes such as ANN relatively provide lesser value in contrast to DLMNN.

Describing the contents and activities in an image or video in semantically and syntactically correct sentences are known as captioning. Automated captioning is one of the most researched topics these days, with new sophisticated models being discovered every day. Captioning models require intense training and perform intense, complex calculations before successfully generating a caption and hence, takes a considerable amount of time even in machines with high specifications. In this survey, we go through the recent state-of-the-art advancements in automatic image and video description methodologies using deep neural networks and summarize the concepts inferred from them. The summarization has been done with a systematic, detailed, and critical analysis of the latest methodologies published in high impact proceedings and journals. Our investigation focuses on techniques that can optimize existing concepts and incorporate new methods of visual attention for generating captions. This survey emphasizes on the importance of applicability and effectiveness of existing works in real-life applications and highlights those computationally feasible and optimized techniques which can be supported in multiple devices, including lightweight devices like smartphones. Furthermore, we propose possible improvements and model architecture to support online video captioning.

This paper proposes a computationally inexpensive method for automatic key-event extraction and subsequent summarization of sports videos using scoreboard detection. A database consisting of 1300 images was used to train a supervised-learning based object detection algorithm, YOLO (You Only Look Once). Then, for each frame of the video, once the scoreboard was detected using YOLO, the scoreboard was cropped out of the image. After this, image processing techniques were applied on the cropped scoreboard to reduce noise and false positives. Finally, the processed image was passed through an OCR (Optical Character Recognizer) to get the score. A rule-based algorithm was run on the output of the OCR to generate the timestamps of key-events based on the game. The proposed method is best suited for people who want to analyse the games and want precise timestamps of the occurrence of important events. The performance of the proposed design was tested on videos of Bundesliga, English Premier League, ICC WC 2019, IPL 2019, and Pro Kabaddi League. An average F1 Score of 0.979 was achieved during the simulations. The algorithm is trained on five different classes of three separate games (Soccer, Cricket, Kabaddi). The design is implemented using python 3.7.

Multimodal summarization aims to extract the most important information from the multimedia input. It is becoming increasingly popular due to the rapid growth of multimedia data in recent years. There are various researches focusing on different multimodal summarization tasks. However, the existing methods can only generate single-modal output or multimodal output. In addition, most of them need a lot of annotated samples for training, which makes it difficult to be generalized to other tasks or domains. Motivated by this, we propose a unified framework for multimodal summarization that can cover both single-modal output summarization and multimodal output summarization. In our framework, we consider three different scenarios and propose the respective unsupervised graph-based multimodal summarization models without the requirement of any manually annotated document-summary pairs for training: (1) generic multimodal ranking, (2) modal-dominated multimodal ranking, and (3) non-redundant text-image multimodal ranking. Furthermore, an image-text similarity estimation model is introduced to measure the semantic similarity between image and text. Experiments show that our proposed models outperform the single-modal summarization methods on both automatic and human evaluation metrics. Besides, our models can also improve the single-modal summarization with the guidance of the multimedia information. This study can be applied as the benchmark for further study on multimodal summarization task.

The era of video data has fascinated users into creating, processing, and manipulating videos for various applications. Voluminous video data requires higher computation power and processing time. In this work, a model is developed that can precisely acquire keyframes through hierarchical summarization and use the keyframes to detect faces and assess the emotional intent of the user. The key-frames are used to detect faces using recursive Viola-Jones algorithm and an emotional analysis for the faces extracted is conducted using an underlying architecture developed based on Deep Neural Networks (DNN). This work has significantly contributed in improving the accuracy of face detection and emotional analysis in non-redundant frames. The number of frames selected after summarization was less than 30% using the local minima extraction. The recursive routine introduced for face detection reduced false positives in all the video frames to lesser than 2%. The accuracy of emotional prediction on the faces acquired through the summarized frames, on Indian faces achieved a 90%. The computational requirement scaled down to 40% due to the hierarchical summarization that removed redundant frames and recursive face detection removed false localization of faces. The proposed model intends to emphasize the importance of keyframe detection and use them for facial emotional recognition.

The volume of video data generated has seen an exponential growth over the years and video summarization has emerged as a process that can facilitate efficient storage, quick browsing, indexing, fast retrieval and quick sharing of the content. In view of the vast literature available on different aspects of video summarization approaches and techniques, a need has arisen to summarize and organize various recent research findings, future research focus and trends, challenges, performance measures and evaluation and datasets for testing and validations. This paper investigates into the existing video summarization frameworks and presents a comprehensive view of the existing approaches and techniques. It highlights the recent advances in the techniques and discusses the paradigm shift that has occurred over the last two decades in the area, leading to considerable improvement. Attempts are made to consolidate the most significant findings right from the basic summarization structure to the classification of summarization techniques and noteworthy contributions in the area. Additionally, the existing datasets categorized domain-wise for the purpose of video summarization and evaluation are enumerated. The present study would be helpful in: assimilating important research findings and data for ready reference, identifying groundwork and exploring potential directions for further research.

Video surveillance has become a major tool in security maintenance. But analyzing in a playback version to detect any motion or any sort of movements might be tedious work because only for a short length of the video there would be any motion. There would be a lot of time wasted in analyzing the video and also it is impossible to always find the accurate frame where the transition has occurred. So there is a need in obtaining a summary video that captures any changes/motion. With the advancements in image processing using OpenCV and deep learning, video summarization is no longer an impossible work. Captions are generated for the summarized videos using an encoder–decoder captioning model. With the help of large, well-labeled video data sets like common objects in context, Microsoft video description, video captioning is a feasible task. Encoder–decoder models are used extensively to extract text from visual features with the arrival of long short term memory (LSTM). Attention mechanism has been widely used on decoder for the work of video captioning. Keyframes are obtained from very long videos using methods like dynamic mode decomposition, an algorithm in fluid dynamics, OpenCV’s absdiff(). We propose these tools for motion detection and video/image captioning for very long videos which are common in video surveillance.

In online systems, including e-commerce platforms, many users resort to the reviews or comments generated by previous consumers for decision making, while their time is limited to deal with many reviews. Therefore, a review summary, which contains all important features in user-generated reviews, is expected. In this article, we study “how to generate a comprehensive review summary from a large number of user-generated reviews.” This can be implemented by text summarization, which mainly has two types of extractive and abstractive approaches. Both of these approaches can deal with both supervised and unsupervised scenarios, but the former may generate redundant and incoherent summaries, while the latter can avoid redundancy but usually can only deal with short sequences. Moreover, both approaches may neglect the sentiment information. To address the above issues, we propose comprehensive Review Summary Generation frameworks to deal with the supervised and unsupervised scenarios. We design two different preprocess models of re-ranking and selecting to identify the important sentences while keeping users’ sentiment in the original reviews. These sentences can be further used to generate review summaries with text summarization methods. Experimental results in seven real-world datasets (Idebate, Rotten Tomatoes Amazon, Yelp, and three unlabelled product review datasets in Amazon) demonstrate that our work performs well in review summary generation. Moreover, the re-ranking and selecting models show different characteristics.

Natural language summaries of codes are important during software development and maintenance. Recently, deep learning-based models have achieved good performance on automatic code summarization, which encode token sequence or abstract syntax tree (AST) of code with neural networks. However, almost all of these models are trained using maximum likelihood estimation, which do not guarantee the quality of generated summaries. Moreover, existing models that benefit from multiple encoders lack a fined-grained selection between different encoders, and the encoders may be insufficiently optimized. To address these issues and generate better code summaries, we propose a novel code summarization framework based on adversarial training and ensemble learning. It includes two separately trained encoder-decoder models, one for source code sequence and the other for its AST. Here, an efficient approach to obtain AST node sequence is introduced. We train our models via adversarial training, where each model is guided by a well-designed discriminator that learns to evaluate its outputs. During inference, a module named mixture network is introduced to compute an adaptive combination weight of the models’ outputs. We evaluate our framework on a large Java corpus and compare it to several state-of-the-art models. Experimental results show that our approach outperforms the best baseline by 22.6% on BLEU-4, 5.7% on ROUGE-L and 7.6% on METEOR.

Sentence boundary detection (SBD) is an essential step for a very large number of natural language processing applications such as parsing, information retrieval, automatic summarization, machine translation, etc. In this paper, we tackle the problem of SBD of dialectal Arabic, especially for the Tunisian dialect. We compare the efficiency of three learning algorithms: Deep Neuronal Networks (DNN), Support Vector Machines (SVM) and Conditional Random Fields (CRF) to detect the boundaries of sentences written in different types of dialect. The best model achieved an F-measure of 84.37% using CRF which is a popular formalism for structured prediction in NLP and it has been widely applied in text segmentation.

Summarization is important in sports video analysis; it gives a more compact and interesting representation of content. The automatic cricket video summarization is more challenging as it contains several rules and longer match duration. In this research, a hybrid machine learning approach is proposed to summarize cricket video. It analyzes the excitement, object, and event-based features for the detection of key events from the cricket video. First, the audio is analyzed for the extraction of the exciting clips by using an adaptive threshold, speech-to-text framework, and Stacked Gated Recurrent Neural Network with Attention Module (SGRNN-AM). Then, the scenes of each exciting clip are classified with a new Hybrid Rotation Forest Deep Belief Network (HRF-DBN). Next, the characters and action features are extracted from the scorecard region of each key frame and umpire frames of exciting clips. Finally, SGRNN-AM model is used to detect key events including fours, sixes, and wickets. The accuracy of the proposed SGRNN-AM video summarization model is increased with an attention module in the hidden outputs of Gated Recurrent Unit (GRU) for selecting the significant features. The performance of the suggested technique has been improved on various collections of cricket videos. It achieved a precision of $$96.82\ \%$$ and an accuracy of $$96.32\%$$ that proves its effectiveness.

This article studies supervised video summarization by formulating it into a sequence-to-sequence learning framework, in which the input and output are sequences of original video frames and their predicted importance scores, respectively. Two critical issues are addressed in this article: short-term contextual attention insufficiency and distribution inconsistency. The former lies in the insufficiency of capturing the short-term contextual attention information within the video sequence itself since the existing approaches focus a lot on the long-term encoder-decoder attention. The latter refers to the distributions of predicted importance score sequence and the ground-truth sequence is inconsistent, which may lead to a suboptimal solution. To better mitigate the first issue, we incorporate a self-attention mechanism in the encoder to highlight the important keyframes in a short-term context. The proposed approach alongside the encoder-decoder attention constitutes our deep attentive models for video summarization. For the second one, we propose a distribution consistency learning method by employing a simple yet effective regularization loss term, which seeks a consistent distribution for the two sequences. Our final approach is dubbed as Attentive and Distribution consistent video Summarization (ADSum). Extensive experiments on benchmark data sets demonstrate the superiority of the proposed ADSum approach against state-of-the-art approaches.\ud

Evolution of the Internet of Things (IoT) makes a revolution in connecting, monitoring, controlling, and managing things, objects, and almost surroundings through the Internet. To reveal the potential of IoT, rich knowledge has to be extracted, indexed, and shared securely in real time. Recent comprehensive researches on IoT spot the light on main correlative challenges, such as security, scalability, heterogeneity, and big data. Due to the heterogeneity of IoT applications that produce a large volume of a variety of data streams in real time, mining, securing, and analyzing IoT data become tedious and challenging tasks. Indexing sensory data is one of data mining techniques, which ease information retrieval. But ordinary indexing methods are not fit with such massive and dynamic data; where indexes become out-of-date once they are built. Clustering, data reduction, and summarization present promising solutions for enabling low-power security and balanced indexing. This article presents a novel method for dynamic data reduction and summarization using dynamic time warping (DTW), which also presents a balanced architecture for enabling balanced indexing based on similarity data fusion. Data reduction-based prediction models enable real-time search and secure discovery for Smart Things (SThs). The results of the proposed model were proved using real examples and data sets. Using the Szeged-weather data set similar SThs data is reduced by 95%. Thus, indexes sizes could be reduced, and using smart scheduling, crawling cycle length could be expanded.

Although a recurrent neural network (RNN) has achieved tremendous advances in video summarization, there are still some problems remaining to be addressed. In this article, we focus on two intractable problems when applying an RNN to video summarization: first the extremely large feature-to-hidden matrices. Since video features are usually in a high-dimensional space, it leads to extremely large feature-to-hidden mapping matrices in the RNN model, which increases the training difficulty. Second, the deficiency in long-range temporal dependence exploration. Most videos contain thousands of frames at least, which is such a long sequence that traditional RNNs cannot deal well with. Facing the abovementioned two problems, we develop a tensor-train hierarchical recurrent neural network (TTH-RNN) for the video summarization task. It contains a tensor-train embedding layer to avert the large feature-to-hidden matrices, together with a hierarchical structure of an RNN to explore the long-range temporal dependence among video frames. Practically, the experimental results on four benchmark datasets, including SumMe, TVsum, MED, and VTW, have demonstrated the excellent performance of a TTH-RNN in video summarization.

When observing the visual world, temporal phenomena are ubiquitous: people walk, cars drive, rivers flow, clouds drift, and shadows elongate. Some of these, like water splashing and cloud motion, occur over time intervals that are either too short or too long for humans to easily observe. High-speed and timelapse videos provide a popular and compelling way to visualize these phenomena, but many real-world scenes exhibit motions occurring at a variety of rates. Once a framerate is chosen, phenomena at other rates are at best invisible, and at worst create distracting artifacts. In this article, we propose to automatically normalize the pixel-space speed of different motions in an input video to produce a seamless output with spatiotemporally varying framerate. To achieve this, we propose to analyze scenes at different timescales to isolate and analyze motions that occur at vastly different rates. Our method optionally allows a user to specify additional constraints according to artistic preferences. The motion normalized output provides a novel way to compactly visualize the changes occurring in a scene over a broad range of timescales.

Peer reviews form an essential part of scientific communications. Research papers and proposals are reviewed by several peers before they are finally accepted or rejected for publication and funding, respectively. With the steady increase in the number of research domains, scholarly venues (journal and/or conference), researchers, and papers, managing the peer review process is becoming a daunting task. Application of recommender systems to assist peer reviewing is, therefore, being explored and becoming an emerging research area. In this paper, we present a deep learning network based Meta-Review Generation considering peer review prediction of the scholarly article (MRGen). MRGen is able to provide solutions for: (i) Peer review prediction (Task 1) and (ii) Meta-review generation (Task 2). First, the system takes the peer reviews as input and produces a draft meta-review. Then it employs an integrated framework of convolution layer, long short-term memory (LSTM) model, Bi-LSTM model, and attention mechanism to predict the final decision (accept/reject) of the scholarly article. Based on the final decision, the proposed model MRGen incorporates Pointer Generator Network-based abstractive summarization to generate the final meta-review. The focus of our approach is to give a concise meta-review that maximizes information coverage, coherence, readability and also reduces redundancy. Extensive experiments conducted on the PeerRead dataset demonstrate good consistency between the recommended decisions and original decisions. We also compare the performance of MRGen with some of the existing state-ofthe- art multi-document summarization methods. The system also outperforms a few existing models based on accuracy, Rouge scores, readability, non-redundancy, and cohesion.

Modern soccer increasingly places trust in visual analysis and statistics rather than only relying on the human experience. However, soccer is an extraordinarily complex game that no widely accepted quantitative analysis methods exist. The statistics collection and visualization are time consuming which result in numerous adjustments. To tackle this issue, we developed GreenSea, a visual-based assessment system designed for soccer game analysis, tactics, and training. The system uses a broad learning system (BLS) to train the model in order to avoid the time-consuming issue that traditional deep learning may suffer. Users are able to apply multiple views of a soccer game, and visual summarization of essential statistics using advanced visualization and animation that are available. A marking system trained by BLS is designed to perform quantitative analysis. A novel recurrent discriminative BLS (RDBLS) is proposed to carry out long-term tracking. In our RDBLS, the structure is adjusted to have better performance on the binary classification problem of the discriminative model. Several experiments are carried out to verify that our proposed RDBLS model can outperform the standard BLS and other methods. Two studies were conducted to verify the effectiveness of our GreenSea. The first study was on how GreenSea assists a youth training coach to assess each trainee's performance for selecting most potential players. The second study was on how GreenSea was used to help the U20 Shanghai soccer team coaching staff analyze games and make tactics during the 13th National Games. Our studies have shown the usability of GreenSea and the values of our system to both amateur and expert users.