Search

This paper explores the use of attributes for document image querying and retrieval. Existing document image retrieval techniques present several drawbacks: textual searches are limited to text, query-by-example searches require a sample query document on hand, and layout-based searches rigidly assign documents to one of several preset classes. Attributes have yet to be fully exploited in document image analysis. We describe document images based on attributes and utilize those descriptions to form a new querying paradigm for document image retrieval that addresses the above limitations: attribute-based document image retrieval (ABDIR). We create attribute-based descriptions of the documents using an expandable set of individual, independent attribute classifiers built on convolutional neural network architectures. We combine the descriptions to form queries of variable complexity which retrieve a ranked list of document images. ABDIR allows users to search for documents based on memorable visual features of their contents in a flexible way, with queries like “Find documents that have a one-column layout, are table dominant, and are colorful”, or “Find historical documents that are illuminated and have see-through artifacts”. Experiments on the recent PubLayNet and HisIR19 datasets demonstrate the system’s ability to extract various document image attributes with high accuracy, with Darknet-53 performing best, and show very promising results for document image retrieval. ABDIR is scalable and versatile: it is easy to change, add, and remove attributes, and easy to adapt queries to new domains. It provides for document image retrieval capabilities that are not possible or are impractical with other paradigms.

Echosounders are used by fisheries and ocean observatories, but significant manual effort is required to classify species of interest within multifrequency echograms. This article investigates the use of modified U-Net convolutional neural networks for the pixel-level classification of biological and physical data in echogram images with accurate classification of herring and salmon schools, bubbles, and the sea surface. Data were collected on the coast of British Columbia, Canada, over two years using an Acoustic Zooplankton and Fish Profiler at four frequencies (67, 125, 200, 455 kHz). In addition, simulated data (water depth and solar elevation angle) provide spatial and temporal context to improve the quality of predictions. Redundancy is built into the model by using a tiling strategy during training and classification. During training, using a limited set of annotated data, translational augmentation encodes the U-Nets with robust features that enable applications for alternate deployment configurations (lower sampling rates or alternate water depths). To ensure broad applicability, these networks were trained to classify echograms with noise left intact. The best-performing model classifies herring, salmon, and bubble classes with $\rm {F_{1}}$ scores of 93.0%, 87.3%, and 86.5%, respectively. The results are accurate even when multiple classes are in close proximity, thus, retaining biological data that would otherwise be discarded due to surface bubble noise.

This paper addresses the detection of finfish and krill in echograms. Finfish, in particular Pacific hake, are used both as human food and fish meal. Krill, harvested for aquaculture and aquariums, are a primary food source for finfish, including hake. Thus, spatial distributions of hake follow that of krill. Stock assessments need an accurate differentiation of krill from finfish (hake) in acoustic echograms. This paper proposes a semantic segmentation paradigm for the pixel-level classification of multi-frequency information to detect co-occurring finfish and krill. This paradigm is highly relevant for identifying cloud-like, diffuse krill aggregations that are intertwined with small, often sparse and sometimes dense schools of finfish. We propose U-MSAA-Net, a deep learning U-Net-like framework with novel multi-scale additive attention (MSAA) modules. MSAA modules allow us to leverage all contextual and local information from feature maps available at any given level of the decoding phase of the network, yielding an efficient suppression of the feature responses from regions with lesser semantic value. Experimental results on a new finfish and krill data set spanning across nine months of acoustic data and covering various situations show that U-MSAA-Net outperforms both traditional, texture-based machine learning methods, and deep learning methods based on state-of-the-art semantic segmentation networks. Additional experiments on a data set containing schools of herring and salmon confirm the versatility of U-MSAA-Net and its superiority in terms of accuracy and ability to detect schools of varying sizes. U-MSAA-Net is the first step in creating a comprehensive tool for stock and ecosystem assessments.

This review paper offers a contemporary literature survey on symbol spotting in architectural drawing images. Research on isolated symbol recognition is quite mature; the same cannot be said for recognizing a symbol in context. One important challenge is the segmentation/recognition paradox: a system should segment symbols before recognizing them, but some kind of recognition may be necessary to obtain a correct segmentation. Research has thus been recently directed toward symbol spotting, a way of locating possible symbol instances without using full recognition methods. In this paper, we thoroughly review symbol spotting methods with a focus on architectural drawings, an application domain providing the document image analysis and graphic recognition communities with an interesting set of challenges linked to the sheer complexity and density of embedded information, that have yet to be resolved. While most existing methods perform well in terms of recall, their performance is rather poor in terms of precision and false positives. In light of the review, we also propose a simple yet effective symbol spotting method based on template matching and a novel clutter-tolerant cross-correlation function that achieves state-of-the-art results with high precision, high recall, and few false positives, able to cope with “real-life clutter” found in industry-standard architectural drawings.