Search

With the goal of solving the problem of feature extractors lacking strong supervision training and insufficient time information concerning single‐sequence model learning, a hierarchical sequence memory network with a multi‐level iterative optimisation strategy is proposed for continuous sign language recognition. This method uses the spatial‐temporal fusion convolution network (STFC‐Net) to extract the spatial‐temporal information of RGB and Optical flow video frames to obtain the multi‐modal visual features of a sign language video. Then, in order to enhance the temporal relationships of visual feature maps, the hierarchical memory sequence network is used to capture local utterance features and global context dependencies across time dimensions to obtain sequence features. Finally, the decoder decodes the final sentence sequence. In order to enhance the feature extractor, the authors adopted a multi‐level iterative optimisation strategy to fine‐tune STFC‐Net and the utterance feature extractor. The experimental results on the RWTH‐Phoenix‐Weather multi‐signer 2014 dataset and the Chinese sign language dataset show the effectiveness and superiority of this method. [ABSTRACT FROM AUTHOR]

To properly maintain and manage track surfaces, the amount of collapsed settlement of a ballast bed must be determined. However, high maintenance costs make it difficult for railway operators to fully maintain and manage track surfaces. Therefore, we propose a method of estimating the shape of the ballast bed using Dense Inverse Search (DIS) optical flow in images captured using a camcorder, which is inexpensive. This paper describes the outline of the method, an evaluation of its accuracy through simulation and model experiments, and the results of applying the method to actual images captured. [ABSTRACT FROM AUTHOR]

High‐dynamic‐range image (HDRI) displays scenes as vivid as the real scenes. HDRI can be reconstructed by fusing a set of bracketed‐exposure low‐dynamic‐range images (LDRI). For the reconstruction, many works succeed in removing the ghost artefacts caused by moving objects. The critical issue is reconstructing the areas which are saturated due to bad exposure and occluded due to motion with no ghost artefacts. To overcome this issue, this paper proposes texture and exposure awareness based refill. The proposed work first locates the saturated and occluded areas existing in input image set, then refills background textures or patches containing rough exposure and colour information into located areas. Proposed work can be integrated with multiple existing ghost removal works to improve the reconstruction result. Experimental results show that proposed work removes the ghost artefacts caused by saturated and occluded areas in subjective evaluation. For the objective evaluation, the proposed work improves the HDR‐VDP‐2 evaluation result for multiple conventional works by 1.33% on average. [ABSTRACT FROM AUTHOR]

Purpose: To develop an image processing methodology for noninvasive three‐dimensional (3D) quantification of microwave thermal ablation zones in vivo using x‐ray computed tomography (CT) imaging without injection of a contrast enhancing material. Methods: Six microwave (MW) thermal ablation procedures were performed in three pigs. The ablations were performed with a constant heating duration of 8 min and power level of 30 W. During the procedure images from sixty 1 mm thick slices were acquired every 30 s. At the end of all ablation procedures for each pig, a contrast‐enhanced scan was acquired for reference. Special algorithms for addressing challenges stemming from the 3D in vivo setup and processing the acquired images were prepared. The algorithms first rearranged the data to account for the oblique needle orientation and for breathing motion. Then, the gray level variance changes were analyzed, and optical flow analysis was applied to the treated volume in order to obtain the ablation contours and reconstruct the ablation zone in 3D. The analysis also included a special correction algorithm for eliminating artifacts caused by proximal major blood vessels and blood flow. Finally, 3D reference reconstructions from the contrast‐enhanced scan were obtained for quantitative comparison. Results: For four ablations located >3 mm from a large blood vessel, the mean dice similarity coefficient (DSC) and the mean absolute radial discrepancy between the contours obtained from the reference contrast‐enhanced images and the contours produced by the algorithm were 0.82 ± 0.03 and 1.92 ± 1.47 mm, respectively. In two cases of ablation adjacent to large blood vessels, the average DSC and discrepancy were: 0.67 ± 0.6 and 2.96 ± 2.15 mm, respectively. The addition of the special correction algorithm utilizing blood vessels mapping improved the mean DSC and the mean absolute discrepancy to 0.85 ± 0.02 and 1.19 ± 1.00 mm, respectively. Conclusions: The developed algorithms provide highly accurate detailed contours in vivo (average error < 2.5 mm) and cope well with the challenges listed above. Clinical implementation of the developed methodology could potentially provide real time noninvasive 3D accurate monitoring of MW thermal ablation in‐vivo, provided that the radiation dose can be reduced. [ABSTRACT FROM AUTHOR]

This paper presents a novel technique for progressive online integration of uncalibrated image sequences with substantial geometric and/or photometric discrepancies into a single, geometrically and photometrically consistent image. Our approach can handle large sets of images, acquired from a nearly planar or infinitely distant scene at different resolutions in object domain and under variable local or global illumination conditions. It allows for efficient user guidance as its progressive nature provides a valid and consistent reconstruction at any moment during the online refinement process. Our approach avoids global optimization techniques, as commonly used in the field of image refinement, and progressively incorporates new imagery into a dynamically extendable and memory‐efficient Laplacian pyramid. Our image registration process includes a coarse homography and a local refinement stage using optical flow. Photometric consistency is achieved by retaining the photometric intensities given in a reference image, while it is being refined. Globally blurred imagery and local geometric inconsistencies due to, e.g. motion are detected and removed prior to image fusion. We demonstrate the quality and robustness of our approach using several image and video sequences, including handheld acquisition with mobile phones and zooming sequences with consumer cameras. [ABSTRACT FROM AUTHOR]

Video endoscopy (VE) and video fluorography (VF) are the classic inspection methods used for evaluating the function of swallowing, but are disadvantageous as they are invasive. Therefore, we have proposed a noninvasive method that uses pharyngeal ultrasound videos to estimate the internal flow characteristics of foods and evaluates the swallowing function level quantitatively. We are proposing a new method that combines optical flow with Maximally Stable Extremal Regions (MSERs) to extract the movement velocity and position of esophagus and bolus. The results showed that the movement velocity could be used as an indicator to quantify the internal flow characteristics of foods. The displacement of the esophagus indicates the esophageal opening; this could be used as an indicator to evaluate the swallowing. In this study, we confirmed that a classification of esophageal and bolus movements should be performed automatically by the developed method to decrease the evaluation time. [ABSTRACT FROM AUTHOR]

In a photo, motion blur can be used as an artistic style to convey motion and to direct attention. In panning or tracking shots, a moving object of interest is followed by the camera during a relatively long exposure. The goal is to get a blurred background while keeping the object sharp. Unfortunately, it can be difficult to impossible to precisely follow the object. Often, many attempts or specialized physical setups are needed. This paper presents a novel approach to create such images. For capturing, the user is only required to take a casually recorded hand‐held video that roughly follows the object. Our algorithm then produces a single image which simulates a stabilized long time exposure. This is achieved by first warping all frames such that the object of interest is aligned to a reference frame. Then, optical flow based frame interpolation is used to reduce ghosting artifacts from temporal undersampling. Finally, the frames are averaged to create the result. As our method avoids segmentation and requires little to no user interaction, even challenging sequences can be processed successfully. In addition, artistic control is available in a number of ways. The effect can also be applied to create videos with an exaggerated motion blur. Results are compared with previous methods and ground truth simulations. The effectiveness of our method is demonstrated by applying it to hundreds of datasets. The most interesting results are shown in the paper and in the supplemental material. [ABSTRACT FROM AUTHOR]

This study presents a human action recognition system from multi-view image sequences. The authors' approach to human action recognition is based on an estimation of local motion from multiple camera views. The authors propose a new motion descriptor, called histogram of motion intensity and direction, to capture local motion characteristics of human activity. After image normalisation, they estimate motion flow using dense optical flow. Using regular grids, they extract local flow motion and estimate the dominant angle and the intensity of optical flow. The histogram of the dominant angle and its intensity are used as a descriptor for each sequence. After the identification of head direction, they concatenate descriptors in each view as a single feature vector from multiple-view sequences. Classification based on the proposed feature vector using support vector machine shows better performance than three-dimensional optical flow-based approaches, but with lower computational requirements. The authors evaluated action recognition on the publicly available i3DPost and the Institut de Recherche en Informatique et en Automatique (INRIA) Xmas Motion Acquisition Sequences database. Experimental results show promising state-of-the-art results and validate the advanced performance of the authors' approach. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) motion analysis of dynamic body organs using volumetric images is of increasing interest in different computer vision applications. A number of methods for estimation of 3D optical flow in those images have been developed in recent years. However, theoretical limits of 3D optical flow-based motion estimation and segmentation are yet to be analysed. In this study, a statistical analysis of 3D optical flow is presented and the results are used to predict the separability of local 3D motions. Experimental results, using both synthetic and real images, demonstrate the applicability of the proposed analysis to predict the separability of two motions in terms of the parameters quantifying their relative motion and the scale of measurement noise. [ABSTRACT FROM AUTHOR]

This paper proposes a global mapping algorithm for multiple robots from an omnidirectional-vision simultaneous localization and mapping (SLAM) approach based on an object extraction method using Lucas-Kanade optical flow motion detection and images obtained through fisheye lenses mounted on robots. The multi-robot mapping algorithm draws a global map by using map data obtained from all of the individual robots. Global mapping takes a long time to process because it exchanges map data from individual robots while searching all areas. An omnidirectional image sensor has many advantages for object detection and mapping because it can measure all information around a robot simultaneously. The process calculations of the correction algorithm are improved over existing methods by correcting only the object's feature points. The proposed algorithm has two steps: first, a local map is created based on an omnidirectional-vision SLAM approach for individual robots. Second, a global map is generated by merging individual maps from multiple robots. The reliability of the proposed mapping algorithm is verified through a comparison of maps based on the proposed algorithm and real maps. [ABSTRACT FROM AUTHOR]

This paper reviews image processing and pattern recognition techniques, which will be useful to analyze bioimages. Although this paper does not provide their technical details, it will be possible to grasp their main tasks and typical tools to handle the tasks. Image processing is a large research area to improve the visibility of an input image and acquire some valuable information from it. As the main tasks of image processing, this paper introduces gray-level transformation, binarization, image filtering, image segmentation, visual object tracking, optical flow and image registration. Image pattern recognition is the technique to classify an input image into one of the predefined classes and also has a large research area. This paper overviews its two main modules, that is, feature extraction module and classification module. Throughout the paper, it will be emphasized that bioimage is a very difficult target for even state-of-the-art image processing and pattern recognition techniques due to noises, deformations, etc. This paper is expected to be one tutorial guide to bridge biology and image processing researchers for their further collaboration to tackle such a difficult target. [ABSTRACT FROM AUTHOR]

In this article, a shape transformation technique is introduced for deforming objects based on a given deformation example. The example consists of two reference shapes representing two different states of an object. The reference shapes are assumed to morph from one state to the other. The evolution between the two reference shapes determines the shape transformation function. Any given objects can then be deformed by the same transformation. A continuous 4D Radial Basis Function is used to construct a density flow field (an extension of the optical flow in computer vision) representing the shape transformation of the example in 3-space. Objects embedded in the density flow field are deformed by moving vertices of the objects along the density flow vectors. Additional parameters are introduced to control the process of the deformation. This provides explicit control on the shape of the object obtained in the deformation process. Copyright © 2007 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

This paper proposes an accurate method for the restoration of missing intensities of still images by using the optical flow. It is important in restoration to reconstruct missing edges correctly. Therefore, this paper modifies the optical flow conventionally used for motion analysis in video images and applies it to the restoration of missing intensities. Further, the proposed method introduces a new index expressing the correlation of intensities between two pixels into the scheme for calculation of the optical flow in order to obtain a flow which gives more accurate estimated values. The optical flow calculated by this index provides the pixel from the neighborhood whose intensity is most similar to that of the target pixel, so that the estimated intensity is not affected by pixels whose intensities are quite different. Consequently, even when multiple edges pass through the missing area or the direction of the edge changes significantly inside the area, the proposed method can reconstruct the edges correctly. Some experimental results are presented in order to verify the high performance of the proposed method. © 2006 Wiley Periodicals, Inc. Syst Comp Jpn, 37(3): 49–57, 2006; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20376 [ABSTRACT FROM AUTHOR]

Kanatani applied the concept of noise to optical flow calculation using the constraint equation proposed by Horn, and presented a renormalization method which estimates the flow and the noise components simultaneously. In the renormalization method, the image is segmented into blocks, and the constraint that flows of the pixels in a block remain the same is assumed. This paper applies MDL criterion to the renormalization method and proposes an optical flow estimation algorithm that segments the image self-organizationally. It is assumed that the sender has a pair of images from time-ordered sequence and the receiver has one of these images. In this assumption, the image segmentation is determined by using MDL criterion so that the intensity gradient vectors generated from the image pair are encoded most efficiently. Each intensity gradient vector is encoded to two components, namely, the flow vector and the error. The image segmentation and the flow vectors obtained by this procedure are defined as the estimated optical flows. As an experiment, the algorithm is applied to artificial image pairs and a real image pair and the results are presented. © 2004 Wiley Periodicals, Inc. Syst Comp Jpn, 35(9): 51–59, 2004; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.10335 [ABSTRACT FROM AUTHOR]

Image registration is very important in analyzing multitemporal images. We have proposed an automated method, ARTSIM (Automated Registration meThod for Satellite IMages), for generating control point pairs on multitemporal images. Since we used spatial correlation as the index of point pair correspondence, the positions and numbers of the control point pair depended on the image quality. This tended to generate fewer pairs of control points in regions having larger spatial discrepancy and to yield insufficient registration accuracy. In this paper, we propose a new method, ARTOFL (Automated Registration Technique using Optical FLow), which measures local disparities between images to be registered and generates the minimum number of control point pairs in adequate positions so as to achieve the desired accuracy of the registration. In the method, we regard the two images to be registered as sequential shots in a movie scene. Optical flow is introduced as a tool for measuring local disparities. A new control point is generated at the point having the maximum length of the optical flow vector. Since images may have severe distortion and/or temporal changes in pixel density, some compensations are required so that larger disparities yield larger optical flow vectors. These procedures are repeated until the maximum length of the optical flow vector falls below a threshold, so that the accuracy of the registration is ensured. The validity of ARTOFL was confirmed by numerical simulation and processing of real remote sensing images. ARTOFL generated fewer control point pairs but provided higher accuracy than ARTSIM. © 2002 Wiley Periodicals, Inc. Electron Comm Jpn Pt 1, 86(1): 38–47, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecja.10034 [ABSTRACT FROM AUTHOR]

The pixel distribution density of human vision is higher near the fovea, becoming lower toward the periphery. It is also reported that the optical flow has an important role in understanding the surrounding situation. This paper proposes the eye movement model based on fovea vision, short-term memory of the optical flow, and parallel execution of the tasks. In the realization of the eye movement while driving in an environment where there are few vehicles, both the narrow-range eye movement to shift the viewpoint to the next edge along the road, and the wide-range eye movement to understand the environmental situation, must be realized. In order to adjust the range of eye movement according to the task, short-term memory depending on the task is considered. It is shown that a wide range of view field can be realized, by using the short-term memory image, which is generated based on the low-level features derived from the optical flow. The eye movement while driving in a situation where there are few vehicles is actually simulated in a simple way, and the usefulness of the proposed model is demonstrated. © 2002 Wiley Periodicals, Inc. Syst Comp Jpn, 33(13): 24–34, 2002; Published online in Wiley InterScience (www.interscience. wiley.com). DOI 10.1002/scj.10132 [ABSTRACT FROM AUTHOR]

This paper presents a method in which moving obstacles such as pedestrians, bicycles, and automobiles are detected from a moving vehicle using images, and the locations of the obstacles are calculated stably in relation to the vehicle. In the proposed method, stereoptic images are projected on the estimated road plane, and a free-space map is constructed without attempting matching between the stereoptic images. Then, the vanishing point of the motion (FOE) is estimated from the optical flow, and the moving obstacle area is extracted from the image based on the residual error. By projecting the moving obstacle area onto the free-space map, the location of the moving obstacle relative to the moving vehicle is determined. In this method, neither tracking between frames nor matching search between stereoptic images is needed, and the location can be determined stably, even for objects with rapidly varying size in the image, or nonrigid objects with continuously changing shape. An experiment was performed in which the locations of moving obstacles such as pedestrians, bicycles, and automobiles were determined from the input image sequence obtained from a camera on a moving vehicle. The effectiveness of the proposed method was demonstrated. © 2001 Scripta Technica, Syst Comp Jpn, 32(14): 67–76, 2001 [ABSTRACT FROM AUTHOR]

We propose a method of detection of moving objects which can work effectively for the scenes having a background with considerable changes. We use the fact that the part which corresponds to the moving object in the scene is uniform in space and has smooth movement. The optical flow is extracted, the region having uniform flow in space is cut, and the region thus obtained is assumed to be the moving object (or a part of it). Then by performing voting on predicted position, objects moving horizontally across the scene are detected as the moving object. According to the intensity of background variation, the threshold value of voting for adaptive detection is determined. We describe the method of detection of a moving object with the constraint that it moves in a straight line with uniform speed and attempt its extension to accelerated motion. © 1999 Scripta Technica, Syst Comp Jpn, 30(11): 107–116, 1999 [ABSTRACT FROM AUTHOR]

In previous analysis of the motion image, the existence of the light source has not particularly been considered. In this paper, the optical model describing the relations among the light source, the object and the view point is introduced. The optical flow estimation problem is discussed based on the gradient method, using the constructed model. First, the relation between the light source and the object model is described as an optical model, and the constraint inherent in the considered scene is derived. As a result, it is shown that the optical flow can be estimated with a high precision also for the scene containing a light source. It is shown that the parameter, which has been considered as a correction term for improving the precision of the optical flow, is a physical parameter indicating the geometrical relation between the light source and the object, from the standpoint of the optical model, Then, not only can the optical flow be estimated with a high precision, but also the information concerning the incident direction from the light source and the reflection coefficients of the object can be newly restored. If the scene is of a simple structure, all parameters that describe the scene, i.e., the information concerning the location of the light source, the object and the camera, as well as the information concerning the shape of the object, and the reflection coefficient of the surface of the object can be estimated. The feasibility of the scene reconstruction is investigated using the CG techniques. [ABSTRACT FROM AUTHOR]

To obtain motion parameters from a moving object in a time-varying image or to obtain information on a three-dimensional structure and the collocation of an object, it is necessary to obtain the displacement from the object on the image prior to further processing. For this purpose, the gradient-based method by which the motion of the object on the image can be detected directly form the spatiotemporal changes of its brightness is widely used. However, in cases where the image contains noise or the conditions required for the gradient-based method are not satisfied, a Large error occurs in the estimation of the object. This paper proposes a method in which the optical flow is represented using polar coordinates, and all the conditions required for its magnitude and argument are considered when they are derived. The magnitude of the optical flow of a pixel and the correlation between its argument and neighboring pixels are examined using true motion. The method detects only the optical flow caused by true motion of an object, eliminating optical flows caused by noise, so that the gradient-based method may be applied. [ABSTRACT FROM AUTHOR]

This paper is concerned with three main problems of image processing occuring on temporal sequences of satellite oceanographic images: approximate localization of the interesting structures on the images; segmentation or determination of the boundary of the structures; and temporal tracking of these boundaries to illustrate their evolution. In this paper, application is done on vortices and results are displayed all over the paper. For this purpose, we propose a new method for optical flow computation and interpretation and a geometric modelling of the structures. Oceanographic images obtained from environmental satellite platforms present a new challenge in computer science. The huge amount of data collected each day and the need for characterizing some specific structures on these images for oceanographic monitoring Justify our approach for the detection and tracking of vortices on oceanographic images. [ABSTRACT FROM AUTHOR]