Search

We describe a method for symbol spotting in born-digital architectural floor plans, which are a particularly challenging application domain of graphics recognition. We propose a hybrid method that capitalizes on strengths of both vector-based and pixel-based symbol spotting techniques. In the description phase, the salient geometric constituents of a symbol are extracted by a variety of vectorization techniques, including a proposed voting-based algorithm for finding partial ellipses. This enables us to better handle local shape irregularities and boundary discontinuities, as well as partial occlusion and overlap. In the matching phase, the spatial relationship between the geometric primitives is encoded via a primitive-aware proximity graph. A statistical approach is then used to rapidly yield a coarse localization of symbols within the plan. Localization is further refined with a pixel-based step implementing a modified cross-correlation function. Experimental results on the public SESYD synthetic dataset demonstrate that our approach clearly outperforms other popular symbol spotting approaches. Moreover, our approach yields promising results on real-world images, which are significantly more challenging in terms of overlap, occlusion, and presence of non-symbolic graphical data.

This review paper offers a contemporary literature survey on symbol spotting in architectural drawing images. Research on isolated symbol recognition is quite mature; the same cannot be said for recognizing a symbol in context. One important challenge is the segmentation/recognition paradox: a system should segment symbols before recognizing them, but some kind of recognition may be necessary to obtain a correct segmentation. Research has thus been recently directed toward symbol spotting, a way of locating possible symbol instances without using full recognition methods. In this paper, we thoroughly review symbol spotting methods with a focus on architectural drawings, an application domain providing the document image analysis and graphic recognition communities with an interesting set of challenges linked to the sheer complexity and density of embedded information, that have yet to be resolved. While most existing methods perform well in terms of recall, their performance is rather poor in terms of precision and false positives. In light of the review, we also propose a simple yet effective symbol spotting method based on template matching and a novel clutter-tolerant cross-correlation function that achieves state-of-the-art results with high precision, high recall, and few false positives, able to cope with “real-life clutter” found in industry-standard architectural drawings.

This paper addresses the automatic identification of pelagic species in acoustic backscatter data. Large quantities of data acquired during underwater acoustic surveys for environmental monitoring and resources management, visualized as echograms, are typically analyzed manually or semi-automatically by marine biologists, which is time-consuming and prone to errors and inter-expert disagreements. In this paper, we propose to detect pelagic species (schools of herring and of juvenile salmon) from echograms with a deep learning (DL) framework based on instance segmentation, allowing us to carefully study the acoustic properties of the targets and to address specific challenges such as close proximity between schools and varying size. Experimental results demonstrate our system’s ability to correctly detect pelagic species from echograms and to outperform an existing object detection framework designed for schools of herring in terms of detection performance and computational resources utilization. Our pixel-level detection method has the advantage of generating a precise identification of the pixel groups forming each detection, opening up many possibilities for automatic biological analyses.

Chest radiographs are used for the diagnosis of multiple critical illnesses (e.g., Pneumonia, heart failure, lung cancer), for this reason, systems for the automatic or semi-automatic analysis of these data are of particular interest. An efficient analysis of large amounts of chest radiographs can aid physicians and radiologists, ultimately allowing for better medical care of lung-, heart- and chest-related conditions. We propose a novel Discrete Wavelet Transform (DWT)-based method for the efficient identification and encoding of visual information that is typically lost in the down-sampling of high-resolution radiographs, a common step in computer-aided diagnostic pipelines. Our proposed approach requires only slight modifications to the input of existing state-of-the-art Convolutional Neural Networks (CNNs), making it easily applicable to existing image classification frameworks. We show that the extra high-frequency components offered by our method increased the classification performance of several CNNs in benchmarks employing the NIH Chest-8 and ImageNet-2017 datasets. Based on our results we hypothesize that providing frequency-specific coefficients allows the CNNs to specialize in the identification of structures that are particular to a frequency band, ultimately increasing classification performance, without an increase in computational load. The implementation of our work is available at github.com/DeclanMcIntosh/LeGallCuda., Published in 2021 18th Conference on Robots and Vision (CRV). 8 pages with referances

Purpose: Prosthetic heart valve designs must be rigorously tested using cardiovascular equipment. The valve orifice area over time constitutes a key quality metric which is typically assessed manually, thus a tedious and error-prone task. From a computer vision viewpoint, a major unsolved issue lies in the orifice being partly occluded by the leaflets’ inner side or inaccurately depicted due to its transparency. Here, we address this issue, which allows us to focus on the accurate and automatic computation of valve orifice areas. Approach: We propose a segmentation approach based on the detection of the leaflets’ free edges. Using video frames recorded with a high-speed digital camera during in vitro simulations, an initial estimation of the orifice area is first obtained via active contouring and thresholding and then refined to capture the leaflet free edges via a curve transformation mechanism. Results: Experiments on video data from pulsatile flow testing demonstrate the effectiveness of our approach: a root-mean-square error (RMSE) on the temporal extracted orifice areas between 0.8% and 1.2%, an average Jaccard similarity coefficient between 0.933 and 0.956, and an average Hausdorff distance between 7.2 and 11.9 pixels. Conclusions: Our approach significantly outperformed a state-of-the-art algorithm in terms of evaluation metrics related to valve design (RMSE) and computer vision (accuracy of the orifice shape). It can also cope with lower quality videos and is better at processing frames showing an almost closed valve, a crucial quality for assessing valve design malfunctions related to their improper closing.

This paper provides a comprehensive comparative study of traditional, hybrid, and deep learning (DL) methods for detecting marine species in echograms. Acoustic backscatter data obtained from multi-frequency echosounders is visualized as echograms and typically interpreted by marine biologists via manual or semi-automatic methods, which are time-consuming. Challenges related to automatic echogram interpretation are the variable size and acoustic properties of the biological targets (marine life), along with significant inter-class similarities. Our study explores and compares three types of approaches that cover the entire range of machine learning methods. Based on our experimental results, we conclude that an end-to-end DL-based framework, that can be readily scaled to accommodate new species, is overall preferable to other learning approaches for echogram interpretation, even when only a limited number of annotated training samples is available.

Marine vessel traffic is one of the main sources of negative anthropogenic impact upon marine environments. The automatic identification of boats in monitoring images facilitates conservation, research and patrolling efforts. However, the diverse sizes of vessels, the highly dynamic water surface and weather-related visibility issues significantly hinder this task. While recent deep learning (DL)-based object detectors identify well medium- and large-sized boats, smaller vessels, often responsible for substantial disturbance to sensitive marine life, are typically not detected. We propose a detection approach that combines state-of-the-art object detectors and a novel Detector of Small Marine Vessels (DSMV) to identify boats of any size. The DSMV uses a short time series of images and a novel bi-directional Gaussian Mixture technique to determine motion in combination with context-based filtering and a DL-based image classifier. Experimental results obtained on our novel datasets of images containing boats of various sizes show that the proposed approach comfortably outperforms five popular state-of-the-art object detectors. Code and datasets available at https://github.com/tunai/hybrid-boat-detection.

Images captured underwater often suffer from suboptimal illumination settings that can hide important visual features, reducing their quality. We present a novel single-image low-light underwater image enhancer, L 2 UWE, that builds on our observation that an efficient model of atmospheric lighting can be derived from local contrast information. We create two distinct models and generate two enhanced images from them: one that highlights finer details, the other focused on darkness removal. A multi-scale fusion process is employed to combine these images while emphasizing regions of higher luminance, saliency and local contrast. We demonstrate the performance of L 2 UWE by using seven metrics to test it against seven state-of-the-art enhancement methods specific to underwater and low-light scenes.

This papers focuses on symbol spotting on real-world digital architectural floor plans with a deep learning (DL)-based framework. Traditional on-the-fly symbol spotting methods are unable to address the semantic challenge of graphical notation variability, i.e. low intra-class symbol similarity, an issue that is particularly important in architectural floor plan analysis. The presence of occlusion and clutter, characteristic of real-world plans, along with a varying graphical symbol complexity from almost trivial to highly complex, also pose challenges to existing spotting methods. In this paper, we address all of the above issues by leveraging recent advances in DL and adapting an object detection framework based on the You-Only-Look-Once (YOLO) architecture. We propose a training strategy based on tiles, avoiding many issues particular to DL-based object detection networks related to the relative small size of symbols compared to entire floor plans, aspect ratios, and data augmentation. Experiments on real-world floor plans demonstrate that our method successfully detects architectural symbols with low intra-class similarity and of variable graphical complexity, even in the presence of heavy occlusion and clutter. Additional experiments on the public SESYD dataset confirm that our proposed approach can deal with various degradation and noise levels and outperforms other symbol spotting methods., Accepted to CVPR2020 Workshop on Text and Documents in the Deep Learning Era

A back-of-the-napkin idea is typically created on the spur of the moment and captured via a few hand-sketched notes on whatever material is available, which often happens to be an actual paper napkin. This paper explores the preservation of such back-of-the-napkin ideas. Hand-sketched notes, reflecting those flashes of inspiration, are not limited to text; they can also include drawings and graphics. Napkin backgrounds typically exhibit diverse textural and colour motifs/patterns that may have high visual saliency from a low-level vision standpoint. We thus frame the extraction of hand-sketched notes as a background modelling and removal task. We propose a novel document background model based on texture mixtures constructed from the document itself via texture synthesis, which allows us to identify background pixels and extract hand-sketched data as foreground elements. Experiments on a novel napkin image dataset yield excellent results and showcase the robustness of our method with respect to the napkin contents. A texture-based background modelling approach, such as ours, is generic enough to cope with any type of hand-sketched notes.

This paper focuses on the rectification of camera-captured document images with varied layouts of mixed contents. Document images acquired via cameras, including smartphones, are typically plagued by perspective, geometric, and/or rotational distortion that hinders document analysis processes. In this paper, we propose an approach to camera-captured image rectification of text and non-text regions that handles perspective, geometric and rotational distortions present in planar and curled documents, extending a state-of-the-art content-based rectification method. We define surface projections via a three-tiered local transformation model, in which primary curved surface projections are formed from individual text regions, and secondary and tertiary surface projections are formed from non-text regions, resulting in a 'patchwork' combination of surfaces spanning the document image. This transformation model allows us to process document images with varied layouts of mixed contents, including large images and graphics, that also contain some justified text. Experiments and comparisons with a state-of-the-art content-based rectification approach on the public IUPR dataset demonstrate the value of the proposed approach on two levels: 1) a significantly improved rectification performance using standard optical character recognition metrics, along with increased document readability, and 2) an improved range of applicability, i.e. ability to correct document images showing various layouts and content types.

Low levels of lighting in images and videos may lead to poor results in segmentation, detection, tracking, among numerous other computer vision tasks. Deep-sea camera systems, such as those deployed on the Ocean Networks Canada (ONC) cabled ocean observatories, use artificial lighting to illuminate and capture videos of deep-water biological environments. When these lighting systems fail, the resulting images become hard to interpret or even completely useless because of their lighting levels. This paper proposes an effective framework to enhance the lighting levels of underwater images, increasing the number of visible, meaningful features. The process involves the dehazing of images using a dark channel prior and fast guided filters.

This letter addresses the texture classification problem through a pixel-based local binary pattern (LBP) statistics aggregation mechanism. Real-world texture images often present challenges for classification algorithms in terms of intra-class variability due, among others, to variable illumination. The LBP operator, a state-of-the-art texture descriptor, possesses key properties for tackling real-world texture images: discriminative power and invariance against monotonic gray level changes. We propose a novel texture classification approach that increases the robustness of LBP-based methods with respect to any type of intra-class variations. The method locally characterizes each pixel with an LBP code histogram and globally computes the label of a textured image by aggregating pixel labels through a voting process. Our approach can be in principle applied to any LBP version, as it focuses on how statistics are computed from LBP codes. We show that the proposed pixel-based approach improves upon traditional LBP block-based approaches in terms of classification accuracy by up to 5.1 p.p. on the public Outex database for the classic LBP with various neighborhoods as well as for various LBP extensions.

In this paper, we look at the societal effects of computer vision technologies from the perspective of the future minds in computer vision: senior year engineering students. Engineering education has traditionally focused on technical skills and knowledge. Nowadays, the need for educating engineers in socio-technical skills and reflective thinking, especially on the bright and dark sides of the technology they develop, is being recognized. We advocate for the integration of social awareness modules into computer vision courses so that the societal effects of technology can be studied together with the technology itself, as opposed to the often more generic 'impact of technology on society' courses. Such modules provide a venue for students to reflect on the real-world consequences of technology in concrete, practical contexts. In this paper, we present qualitative results of an observational study analyzing essays of senior year engineering students, who wrote about societal impacts of computer vision technologies of their choice. Privacy and security issues ranked as the top impact topics discussed by students among 50 topics. Similar social awareness modules would apply well to other advanced technical courses of the engineering curriculum where privacy and security are a major concern, such as big data courses. We believe that such modules are highly likely to enhance the reflective abilities of engineering graduates regarding societal impacts of novel technologies.

This paper focuses on the automatic quantitative performance analysis of bioprosthetic heart valves from video footage acquired during in vitro testing. Bioprosthetic heart valves, mimicking the shape and functionality of a human heart valve, are routinely used in valve replacement procedures to substitute defective native valves. Their reliability in both functionality and durability is crucial to the patients' well-being, as such, valve designs must be rigorously tested before deployment. A key quality metric of a heart valve design is the cyclical temporal evolution of the valve's area. This metric is typically computed manually from input video data, a time-consuming and error-prone task. We propose a novel, cost-effective approach for the automatic tracking and segmentation of valve orifices that integrates a probabilistic motion boundary model into a distance regularized level set evolution formulation. The proposed method constrains the level set evolution domain using data about characteristic motion patterns of heart valves. Experiments including comparisons with two other methods demonstrate the value of the proposed approach on three levels: an improved segmented orifice shape accuracy, a greater computational efficiency, and a better ability to identify video frames with orifice area content (open valve).

Contemporary business documents contain diverse, multi-layered mixtures of textual, graphical, and pictorial elements. Existing methods for document segmentation and classification do not handle well the complexity and variety of contents, geometric layout, and elemental shapes. This paper proposes a novel document image classification approach that distributes individual pixels into four fundamental classes (text, image, graphics, and background) through support vector machines. This approach uses a novel low-dimensional feature descriptor based on textural properties. The proposed feature vector is constructed by considering the sparseness of the document image responses to a filter bank on a multi-resolution and contextual basis. Qualitative and quantitative evaluations on business document images show the benefits of adopting a contextual and multi-resolution approach. The proposed approach achieves excellent results; it is able to handle varied contents and complex document layouts, without imposing any constraint or making assumptions about the shape and spatial arrangement of document elements.

Hand gesture recognition is an expansive and evolving field. Previous work addresses methods for tracking hand gestures with specialty gaming/desktop environments in real time. The method proposed here focuses on enhancing performance for mobile GPU platforms with restricted resources by limiting memory use/transfers and by reducing the need for code branches. An encoding scheme has been designed to allow contour processing typically used for finding fingertips to occur efficiently on a GPU for non-touch, remote manipulation of on-screen images. Results show high resolution video frames can be processed in real time on a modern mobile consumer device, allowing for fine grained hand movements to be detected and tracked.

With the increase in media streaming content and consumer-level video creation, there is a high demand for automatic video summarization systems. This paper proposes a bottom-up approach for the automatic generation of dynamic video summaries. Our approach integrates motion and saliency analysis with temporal slicing to extract features from the video, and to further find candidate shots. A shot similarity measure is proposed for constructing the dynamic summaries for candidate shots. From a practical perspective, our main contribution is the design of a video summarization system that is independent on the video domain. We show that the system performs equally well for domains at the extreme opposites of the domain spectrum, namely professionally edited videos and egocentric videos, without any prior information on the video contents.

Prosthetic heart valves (PHVs) are routinely used in clinical settings to replace defective native heart valves in patients suffering from valvular heart disease. Although PHV designs must be rigorously tested using cardiovascular testing equipment to ensure their optimal characteristics and safe operation, visual data obtained during simulations are typically assessed manually, a tedious and error-prone task. The valve orifice area over time, which informs on the opening and closing motion patterns, constitutes a key quality metric for PHV assessment. In addition to the very fast motion of the valve’s leaflets, a major issue lies in the orifice being partly occluded by the leaflets’ inner side or inaccurately depicted due to its transparency, which is not addressed in the literature. In this paper, we propose a novel orifice segmentation approach for automatic PHV quantitative performance analysis, based on the detection of the leaflet free edges to accurately extract the actual orifice area. Utilizing video frames recorded with a high speed digital camera during in vitro simulations, an initial estimation of the orifice area is first obtained via active contouring and then refined to capture the leaflet free edges via a curve extension scheme based on brightness and smoothness criteria. Evaluation on three different PHVs demonstrated the effectiveness of our approach to detect valve leaflet free edges and extract the actual orifice area, significantly outperforming a baseline algorithm both in terms of valve design evaluation metrics and computer vision evaluation metrics.

Due to the rapidly increasing quality of cameras and processing power in smartphones, citizen scientists can play a more significant role in environmental monitoring and ecological observations. Determining the size of large bird flocks, like those observed during migration seasons, is important for monitoring the abundance of bird populations as wildlife habitats continue to shrink. This paper describes a pilot study aimed at automatically counting birds in large moving flocks, filmed using hand-held devices. Our proposed approach integrates motion analysis and segmentation methods to cluster and count birds from video data. Our main contribution is the design of a bird counting algorithm that requires no human input, and functions well for videos acquired in non-ideal conditions. Experimental evaluation is performed using ground truth of manual annotations and bird counts, and shows promising results.

Nonverbal cues constitute a significant part of human communication. Traditionally the object of psychology, nonverbal communication studies now permeate fields such as social signal processing and human computer interaction. The ubiquity of digital recordings of human social interactions and of free sharing platforms offers many opportunities for the automated analysis of group interaction dynamics; yet, most research relies on multimodal cues and strict setups, which are incompatible with this vast pool of video data. In this paper, we focus on the automatic identification of non-talking participants in videos of panel conversations acquired in uncontrolled environments, based solely on visual nonverbal cues. Our approach characterizes human body motion with a novel feature descriptor based on a non-linear model of pixel change history; motor behavioral patterns derived from this descriptor are then utilized via supervised machine learning to identify non-talking participants in each frame and provide an assessment of the participants' engagement levels. Performance evaluation on a challenging dataset demonstrated the effectiveness of our approach to detect non-speakers, with an overall F-score of 86.2%, as well as its robustness to varied settings. To the best of our knowledge, this is the first attempt at identifying non-talking participants for engagement level assessments from a computer vision viewpoint, which has several relevant applications, such as content-based video retrieval and video summarization.

The concept of sparseness has played an important role in classical signal processing applications such as the acquisition, sampling, and compression of high-dimensionality signals, as well as in various machine learning techniques. Computer vision applications have also benefited in more recent years from sparse representations, which can help recover semantic information from images. In this paper, we shed a unique light on the concept of sparseness and propose a comparative study of four popular sparseness measures applied in a novel way to the problem of texture segmentation. Low-dimensional, contextual, multi-resolution descriptors are derived directly from the sparseness of the pixels' responses to a Gabor filter bank, unlike traditional sparseness-based approaches, we do not impose constraints on or make assumptions about the sparseness of the data or of their representation. Textured images are segmented through pixel labelling via supervised machine learning using the sparseness-based descriptors. The behaviour of the four compared sparseness measures, namely Hoyer's measure, the Gini index, the kurtosis, and the normalized hyperbolic tangent, is analyzed with respect to general rules and desirable attributes using synthetic examples, and is evaluated for texture segmentation problems on the public Outex dataset with respect to texture classes and pixel pattern categories. We found that although the four measures all intend to capture the same information, they yield very different segmentation results, and we recommend the Gini index as the sparseness measure of choice for texture segmentation problems.

We propose a system that automatically detects abnormal, violent actions that are performed by individual subjects and witnessed by passive crowds. The problem of abnormal individual behavior, such as a fight, witnessed by passive bystanders gathered into a crowd has not been studied before. We show that the presence of a passive, standing crowd is an important indicator that an abnormal action might occur. Thus, detecting the standing crowd improves the performance of detecting abnormal, violent actions. The proposed method performs crowd detection first, followed by the detection of abnormal motion events. Our main theoretical contribution consists in linking crowd detection to abnormal, violent actions, as well as in defining novel sets of features that characterize static crowds and abnormal individual actions in both spatial and spatio-temporal domains. Experimental results are computed on a custom dataset, the Vancouver Riot Dataset, that we generated using amateur video footage acquired with handheld devices and uploaded on public social network sites. Our approach achieves a high precision and recall values, which validates our system reliability of localizing the crowds and the abnormal actions.

This paper proposes an approach to compute view-normalized body part trajectories of pedestrians walking on potentially non-linear paths. The proposed approach finds applications in gait modeling, gait biometrics, and in medical gait analysis. Our approach uses the 2D trajectories of both feet and the head extracted from the tracked silhouettes. On that basis, it computes the apparent walking (sagittal) planes for each detected gait half-cycle. A homography transformation is then computed for each walking plane to make it appear as if walking was observed from a fronto-parallel view. Finally, each homography is applied to head and feet trajectories over each corresponding gait half-cycle. View normalization makes head and feet trajectories appear as if seen from a fronto-parallel viewpoint, which is assumed to be optimal for gait modeling purposes. The proposed approach is fully automatic as it requires neither manual initialization nor camera calibration. An extensive experimental evaluation of the proposed approach confirms the validity of the normalization process.

This paper proposes an approach to compute and evaluate view-normalized body part trajectories of pedestrians from monocular video sequences. The proposed approach uses the 2D trajectories of both feet and of the head extracted from the tracked silhouettes. On that basis, it segments the walking trajectory into piecewise linear segments. Finally, a normalization process is applied to head and feet trajectories over each obtained straight walking segment. View normalization makes head and feet trajectories appear as if seen from a fronto-parallel viewpoint. The latter is assumed to be optimal for gait modeling and identification purposes. The proposed approach is fully automatic as it requires neither manual initialization nor camera calibration. An extensive experimental evaluation of the proposed approach confirms the validity of the normalization process.

This paper proposes a new morphology-based approach for the interslice interpolation of current transformer (CT) and MRI datasets composed of parallel slices. Our approach is object based and accepts as input data binary slices belonging to the same anatomical structure. Such slices may contain one or more regions, since topological changes between two adjacent slices may occur. Our approach handles explicitly interslice topology changes by decomposing a many-to-many correspondence into three fundamental cases: one-to-one, one-to-many, and zero-to-one correspondences. The proposed interpolation process is iterative. One iteration of this process computes a transition sequence between a pair of corresponding input slices, and selects the element located at equal distance from the input slices. This algorithmic design yields a gradual, smooth change of shape between the input slices. Therefore, the main contribution of our approach is its ability to interpolate between two anatomic shapes by creating a smooth, gradual change of shape, and without generating over-smoothed interpolated shapes.

This paper proposes a new perceptual interface for the control of computer-based music production. We address the constraints imposed by the use of musical meta-instruments during live performance or rehearsal by tracking feet motion relatively to a visual keyboard. The visual attribute stands for the fact that, unlike its physical counterpart, our keyboard does not involve any force feedback during key-presses. The proposed tracking algorithm is structured on two levels, namely a coarse level for foot regions, and a fine level for foot tips. Tracking works in real-time and handles efficiently feet regions merging/unmerging due to spatial proximity and cast shadows. The output of the tracking is used for the spatiotemporal detection of key-''press'' events.

We propose and evaluate a method for learning deep-sea substrate types using video recorded with a remotely operated vehicle (ROV). The goal of this work is to create a labelled spatial map of substrate types from ROV video in order to support biological and geological domain research. The output of our method describes the mixtures of geological features such as sediment and types of lava flow in images taken at a set of points chosen from an ROV dive. The main contribution of this work is the assembly of a pipeline combining several unique approaches which is able to robustly generate substrate type mixtures under the varying lighting and perspective conditions of deep-sea ROV dive videos. The pipeline comprises three main components: sampling, in which a trained classifier and spatial sampling is used to select relevant frames from the dataset; feature extraction, in which the improved local binary pattern descriptor (ILBP) is used to generate a Bag of Words (BoW) representation of the dataset; and topic modelling in which a variant of Latent Dirichlet Allocation (LDA), is used to infer the mixture of substrate types represented by each BoW. Our method significantly outperforms techniques relying on keypoint based features rather than texture based features, and k-means rather than LDA, demonstrating that our proposed pipeline accurately learns and identifies visible substrate types.

This paper proposes a novel approach for the analysis of movement and behavior of the Plainfin midshipman (Porichthys notatus) in the wild. It is based on underwater video recordings of the fish in their natural habitat taken inside their nests during reproductive months. During this time, alpha male Plainfin midshipmen rarely leave their nests as they are guarding their eggs, so the proposed approach addresses the issue of detecting subtle motion and nesting behavior as the fish remains relatively sedentary. To the best of our knowledge, this is the first paper to propose an automated method to analyze subtle movements of a highly territorial animal in its natural habitat. Motion detection uses the displacement of SURF (Interest point algorithm) key-point movements from frame to frame to analyze the amount of movement by the fish. K-means clustering and other outlier removal techniques are then used to differentiate fish motion from small moving objects in the background and foreground. The analysis of fish behavior uses similarity-based periodicity detection combined with the K-neighbors classifier. Experimental validation with respect to expert-annotated ground truth shows excellent performance for both motion and behavior detection approaches.

This paper describes a new method for the temporal segmentation of periodic human activities from continuous real-world indoor video sequences acquired with a static camera. The proposed approach is based on the concept of inter-frame similarity matrix. Indeed, this matrix contains relevant information for the analysis of cyclic and symmetric human activities, where the motion performed during the first semi-cycle is repeated in the opposite direction during the second semi-cycle. Thus, the pattern associated with a periodic activity in the similarity matrix is rectangular and decomposable into elementary units. We propose a morphology-based approach for the detection and analysis of activity patterns. Pattern extraction is further used for the detection of the temporal boundaries of the cyclic symmetric activities. The approach for experimental evaluation is based on a statistical estimation of the ground truth segmentation and on a confidence ratio for temporal segmentations.

Using video and imagery for assessing the distribution and abundance of marine organisms is a valuable sampling method in that it is non-invasive and permits large volumes of data to be acquired. Quickly and accurately processing large volumes of imagery is a challenge for human analysts, which motivates the need for automated processing methods. In this paper, we present a method for the automatic classification of sea stars in underwater images. The method uses a very small number of features and is efficient. The classification process is optimized by using evolutionary computational methods. Experimental results show excellent performance of our proposed optimized classification approach.

In this paper, we present a challenging dataset for the purpose of segmentation and change detection in photographic images of mountain habitats. We also propose a baseline algorithm for habitats segmentation to allow for performance comparison. The dataset consists of high resolution image pairs of historic and repeat photographs of mountain habitats acquired in the Canadian Rocky Mountains for ecological surveys. With a time lapse of 70 to 100 years between the acquisition of historic and repeat images, these photographs contain critical information about ecological change in the Rockies. The challenging aspects of analyzing these image pairs come mostly from the perspective (oblique) view of the photographs and the lack of color information in the historic photographs. The baseline algorithm that we propose here is based on texture analysis and machine learning techniques. Classifier training and results validation are made possible by the availability of expert manual ground-truth segmentation for each image. The results obtained with the baseline algorithm are promising and serve as a reference for new and improved segmentation and change detection algorithms.

Marine optical imaging has become a major assessment tool in science, policy and public understanding of our seas and oceans. Methodology in this field is developing rapidly, including hardware, software and the ways of their application. The aim of the Marine Imaging Workshop (MIW) is to bring together academics, research scientists and engineers, as well as industrial partners to discuss these developments, along with applications, challenges and future directions. The first MIW was held in Southampton, UK in April 2014. The second MIW, held in Kiel, Germany, in 2017 involved more than 100 attendees, who shared the latest developments in marine imaging through a combination of traditional oral and poster presentations, interactive sessions and focused discussion sessions. This article summarises the topics addressed during the workshop, particularly the outcomes of these discussion sessions for future reference and to make the workshop results available to the open public.

In this paper, we present a non-invasive method of counting fish in their natural habitat using automated analysis of video data. Our approach uses three modular components to preprocess, detect, and track the fish. The preprocessing reduces noise present in the image while enhancing the fish using several different techniques. The fish detection is based on two background subtraction algorithms which are computed independently and later combined with logical operations. The tracking is then carried out by a heuristic blob tracking algorithm. The paper presents a description of the proposed counting method as well as its experimental validation.

This paper exploits the concept of sparseness to generate novel contextual multi-resolution texture descriptors. We propose to extract low-dimension features from Gabor-filtered images by considering the sparseness of filter bank responses. We construct several texture descriptors: the basic version describes each pixel by its contextual textural sparseness, while other versions also integrate multi-resolution information. We apply the novel low-dimension sparseness-based descriptors to the problem of texture segmentation and evaluate their performance on the public Outex database. The sparseness-based descriptors show a substantial improvement over Gabor filters with respect not only to computational costs and memory usage, but also to segmentation accuracy. The proposed approach also shows a desirable smooth, monotonic behavior with respect to the dimensionality of the descriptors.

The identification of pond turtles is important to scientists who monitor local populations, as it allows them to track the growth and health of subjects over their lifetime. Traditional non-invasive methods for turtle recognition involve the visual inspection of distinctive coloured patterns on their plastron. This visual inspection is time consuming and difficult to scale with a potential growth in the surveyed population. We propose an algorithm for automatic identification of individual turtles based on images of their plastron. Our approach uses a combination of image processing and neural networks. We perform a convexity-concavity analysis of the contours on the plastron. The output of this analysis is combined with additional region-based measurements to compute feature vectors that characterize individual turtles. These features are used to train a neural network. Our goal is to create a neural network which is able to query a database of images of turtles of known identity with an image of an unknown turtle, and which outputs the unknown turtle's identity. The paper provides a thorough experimental evaluation of the proposed approach. Results are promising and point towards future work in the area of standardized image acquisition and image denoising.

This paper presents a novel solution for the layout segmentation of graphical elements in Business Intelligence documents. We propose a generalization of the recursive X-Y cut algorithm, which allows for cutting along arbitrary oblique directions. An intermediate processing step consisting of line and solid region removal is also necessary due to presence of decorative elements. The output of the proposed segmentation is a hierarchical structure which allows for the identification of primitives in pie and bar charts. The algorithm was tested on a database composed of charts from business documents. Results are very promising.

This paper describes a computational approach for analyzing and visualizing the aesthetics of color from the perspective of color theory. Our study is grounded in the works of Johannes Itten, one of the most remarkable theorists of color aesthetics. Our focus lies on the computational analysis of a specific aspect of color usage in paintings, namely modulation. We, therefore, propose the three-dimensional (3-D) color palette, a visualization of the chromatic information of an image in the hue-saturation-lightness space. Using the proposed palette, we derive a set of simple hue-specific descriptors for color modulation. Our experimental results involve a selection of digital reproductions of paintings discussed extensively by Itten. They show that the proposed modulation measures yield results that are consistent with Itten’s comments and explanations. Future work involves further exploration of the proposed 3-D color palette, in terms of its ability to discriminate between different artists and painting styles.

This paper describes a novel algorithm for tracking the motion of the urethra from trans-perineal ultrasound. Our work is based on the structure-from-motion paradigm and therefore handles well structures with ill-defined and partially missing boundaries. The proposed approach is particularly well-suited for video sequences of low resolution and variable levels of blurriness introduced by anatomical motion of variable speed. Our tracking method identifies feature points on a frame by frame basis using the SURF detector/descriptor. Inter-frame correspondence is achieved using nearest-neighbor matching in the feature space. The motion is estimated using a non-linear bi-quadratic model, which adequately describes the deformable motion of the urethra. Experimental results are promising and show that our algorithm performs well when compared to manual tracking.

We propose a new method for view-invariant gait modeling using a single calibrated camera. Piecewise-continuous body parts trajectories extracted from a video sequence are rectified to appear as observed from a fronto-parallel view. Standard gait characteristics are then computed by combining rectified gait half-cycles from each trajectory. In this method, we make use of a walk model that allows changes in direction as well as changes in speed in order to decouple gait characteristics from distracting factors in the observed sequence. In contrast with previous work, our method is thus well suited for both clinical and surveillance applications. Simulated and real trajectories from an indoor setting are used to validate the proposed method.

This paper presents a novel paradigm for prototyping Computer Vision algorithms; this paradigm is suitable for students with very limited programming experience. Raydiance includes a tangible user interface controlled by a spatial arrangement of physical tokens which are detected using computer vision techniques. Constructing an algorithm is accomplished by creating a directed graph of token connections. Data is processed, then propagated from one token to another by using a novel Light Ray metaphor. Our case study shows how Raydiance can be used to construct a computer vision algorithm for a particular task.

This study reports on the quantitative evaluation of a set of state-of-the-art feature detectors in the context of repeat photography. Unlike most related work, the proposed study assesses the performance of feature detectors when intra-pair variations are uncontrolled and due to a variety of factors (landscape change, weather conditions, different acquisition sensors). There is no systematic way to model the factors inducing image change. The proposed evaluation is performed in the context of image matching, i.e. in conjunction with a descriptor and matching strategy. Thus, beyond just comparing the performance of these detectors, we also examine the feasibility of feature-based matching on repeat photography. Our dataset consists of a set of repeat and historic images pairs that are representative for the database created by the Mountain Legacy Project www.mountainlegacy.ca.

This paper addresses the problem of tracking human motion in a movie sequence involving camera movement. We have developed an approach to track the bounding box of a human in motion without using any particular model. This method exploits motion vector fields from the image, then subtracts the motion caused by the camera to obtain the segmentation of the object. In addition, we introduce a multilevel tracking approach. This approach makes the tracking operation more robust, and less prone to errors. Experiments with movie sequences representing human walk are reported.

This paper proposes a new method for the automatic contrast enhancement of fiducial markers in low-radiation Electronic Portal Images. It is shown that the proposed approach significantly enhances the contrast of the fiducial markers and produces results where these markers are clearly visible. The main theoretical contribution consists in designing an algorithm that enhances the contrast of small structures in noisy images; the parameters of this algorithm are not empirically selected, but determined via a maximum search over a contrast metric. From a practical standpoint, the proposed method has direct applications in the current clinical workflow involving manual marker detection. It is also able to significantly improve the performances of automatic marker detection reported in literature.

This paper proposes a method to obtain fronto-parallel (side-view) body part trajectories for a walk observed from an arbitrary view. The method is based on homography transformations computed for each gait half-cycle detected in the walk. Each homography maps the body part trajectories to a simulated side view of the walk. The proposed method is stable as the resulting normalized trajectories are not influenced by missing or omitted parts of the raw trajectories. Experiments confirm that normalized trajectories are in agreement with the ones that would be obtained from a side view.

This paper proposes a novel approach for the real-time detection of sleep onset in fatigued drivers. Sleep onset is the most critical consequence of fatigued driving, as shown by statistics of fatigue-related crashes. Therefore, unlike previous related work, we separate the issue of sleep onset from the global analysis of the physiological state of fatigue. This allows us for formulating our approach as an event-detection problem. Real-time performance is achieved by focusing on a single visual cue (i.e. eye-state), and by a custom-designed template-matching algorithm for on-line eye-state detection.

The approach proposed in this paper retrieves contours from transrectal ultrasound (TRUS) prostate images. The input images are sparsely annotated by radiologists for the purpose of brachytherapy planning and post-interventional monitoring. The theoretical contribution of the paper consists in the design of a task-oriented, bottom-up method which mimics perceptual grouping mechanisms for contour retrieval. The proposed approach is task-oriented because it embeds prior anatomical and procedural knowledge. From a practical standpoint, the proposed approach is of clinical relevance, since it allows for retrieving contours from images where the annotation is 'blended' with the image content. While new image annotation systems are able to store image content and annotations in a separate manner, many TRUS prostate databases still contain 'blended' annotations only. Our approach allows for contour retrieval and further 3D prostate modeling from such databases.

This paper describes a new three-dimensional spatiotemporal template, namely the Volumetric Motion History Image (VMHI), for the purpose of human motion analysis. Irregularities in human actions typically occur either in speed or orientation; they carry information about the balance and the confidence level of the human subject performing the activity. The proposed VMHI template handles successfully shortcomings of existing spatiotemporal templates related to motion self -occlusion and speed. Therefore, VMHI allows for interactive visualization, as well as quantification of motion performance. This study focuses on the analysis of sway and speed-related abnormalities, which are among the most common motion irregularities in the studied set of human actions.

This paper presents an integrated region-based and gradient-based supervised method for segmentation of a patient magnetic resonance images (MRI) of the shoulder joint. The method is noninvasive, anatomy-based and requires only simple user interaction. It is generic and easily customizable for a variety of routine clinical uses in orthopedic surgery.

This paper proposes an approach to compute view normalized body part trajectories of pedestrians from monocular video sequences. The proposed approach first extracts the 2D trajectories of both feet and of the head from tracked silhouettes. On that basis, it segments the walking trajectory into piecewise linear segments. Finally, a normalization process is applied to head and feet trajectories over each obtained straight walking segment. View normalization makes head and feet trajectories appear as if seen from a fronto-parallel viewpoint. The latter is assumed to be optimal for gait modeling and recognition purposes. The proposed approach is fully automatic as it requires neither manual initialization nor camera calibration.