Search

Understanding and monitoring the surrounding environment increasingly rely on its 3-D representations. However, the often high costs of 3-D data equipment limit its wide usage, and low-cost solutions are in demand. Here, we propose a novel approach based on spherical stereo videos captured with a known baseline (distance between the cameras) for a low-cost and efficient 3-D point cloud reconstruction. In a forest environment, we evaluated 1) the influence of baseline length on point cloud quality and 2) the suitability of the generated point clouds for extracting primary forest attributes (tree position and diameter). Our results show that the proposed approach allows for feasible 3-D reconstruction of complex forest plots. The highest point cloud quality was achieved with a baseline of 60 cm. This setup enabled the correct detection of more than 65% of the trees within the forest plots, producing an average tree position error between 30 and 50 cm and clearly outperforming other setups. A multiscale model-to-model cloud comparison analysis showed signed distances between the generated point cloud and the reference data with zero mean and 1 m standard deviation. We demonstrate that the proposed approach can be a valuable low-cost solution for 3-D point cloud reconstruction, facilitating forest assessment and monitoring.

Compared with the traditional video, omnidirectional stereo video (ODSV) provides a larger field of view (FOV) with depth perception but makes the capturing, processing and displaying more complicated. Even though many attempts have been made to address these challenges, they leave one or more of the following problems: complicated camera rig, high latency and visible distortions. This paper presents a practical end-to-end solution based on a novel hybrid representation to solve these problems simultaneously. The proposed solution is directly from capturing to displaying, which removes the processing step, thus reducing the total time consumption and visible stitching distortions. This hybrid representation is piecewise linear about the horizontal viewing direction whose domain of definition is 0∘ to 360∘ with an assumption that the background is static, consisting of both static and moving regions. Using this representation, ODSV can be presented by omnidirectional stereo images and normal stereo pair of videos respectively. Moreover, a single panoramic camera strategy can be adopted to capture the omnidirectional stereo images in real environment and a normal binocular camera can be used to capture the stereo pair of videos. To display the ODSV, this paper presents a real-time tracking-based rendering algorithm for head mounted display (HMD). Experiments show that the proposed method is effective and cost-efficient. In contrast to state-of-the-art methods, the proposed method significantly reduces the complexity of camera rig and data amount, preserving a competitive stereo quality without visible distortions. [ABSTRACT FROM AUTHOR]

Abstract The redistribution of marine ecosystem engineers in response to changing climate is restructuring endemic benthic communities globally. Therefore, developing and implementing efficient monitoring programs across the complete depth range of these marine ecosystem engineers is often an urgent management priority. Traditionally, many monitoring programs have been based on a systematically selected set of survey locations that, while able to track trends at those sites through time, lack inference for the overall region being monitored. This study trialled a probabilistic sampling design to address this need, taking advantage of an important prerequisite for such designs, extensive multibeam echosounder (MBES) mapping, to inform a spatially balanced sample selection. Here, we allocated 170 remotely operated vehicles (ROVs) transects based on a spatially balanced probabilistic sampling design across three locations with extensive mapping. Generalized additive models were used to estimate the density and associated barren cover of the range‐expanding ecosystem engineer, the long spined urchin (Centrostephanus rodgersii). Estimates were generated at a reef‐wide scale across three locations on the east coast of Tasmania, Australia, representing the leading edge of the species recent range extension. Model‐based estimates of urchin density and barren cover incorporated seabed structure attributes, such as depth and ruggedness, with differences in these modelled relationships being identified between locations. Estimates ranged from 0.000065 individuals m−2 and 0.018% barren cover in the Tasman Peninsula to 0.167 individuals m−2 and 2.10% barren cover at Governor Island Marine Reserve, reflecting a north to south distributional gradient. This study highlights the value of combining probabilistic sampling designs, ROV transects, stereo video, and MBES mapping to generate reliable and robust estimates of important ecosystem species needed to protect reef‐based fishery and conservation values via adaptive and informed management.

The redistribution of marine ecosystem engineers in response to changing climate is restructuring endemic benthic communities globally. Therefore, developing and implementing efficient monitoring programs across the complete depth range of these marine ecosystem engineers is often an urgent management priority. Traditionally, many monitoring programs have been based on a systematically selected set of survey locations that, while able to track trends at those sites through time, lack inference for the overall region being monitored. This study trialled a probabilistic sampling design to address this need, taking advantage of an important prerequisite for such designs, extensive multibeam echosounder (MBES) mapping, to inform a spatially balanced sample selection. Here, we allocated 170 remotely operated vehicles (ROVs) transects based on a spatially balanced probabilistic sampling design across three locations with extensive mapping. Generalized additive models were used to estimate the density and associated barren cover of the range‐expanding ecosystem engineer, the long spined urchin (Centrostephanus rodgersii). Estimates were generated at a reef‐wide scale across three locations on the east coast of Tasmania, Australia, representing the leading edge of the species recent range extension. Model‐based estimates of urchin density and barren cover incorporated seabed structure attributes, such as depth and ruggedness, with differences in these modelled relationships being identified between locations. Estimates ranged from 0.000065 individuals m−2 and 0.018% barren cover in the Tasman Peninsula to 0.167 individuals m−2 and 2.10% barren cover at Governor Island Marine Reserve, reflecting a north to south distributional gradient. This study highlights the value of combining probabilistic sampling designs, ROV transects, stereo video, and MBES mapping to generate reliable and robust estimates of important ecosystem species needed to protect reef‐based fishery and conservation values via adaptive and informed management. [ABSTRACT FROM AUTHOR]

The enhancement of wave breaking activity during wave group passage is investigated using coherent field observations of the instantaneous sea surface elevation and whitecap coverage from platform‐based stereo video measurements in the central North Sea. Passing wave groups are shown to be associated with a two to threefold enhancement in the probability distribution of total whitecap coverage W whereas the enhancement of active whitecap coverage WA is approximately fivefold. Breaking time scales and intermittency characteristics are also investigated with the inclusion of a secondary data set of W and WA observations collected during a research cruise in the North Pacific. The time scale analysis suggests a universal periodicity in wave breaking activity within a representative sea‐surface area encompassing approximately one dominant wave crest. The breaking periodicity is shown to be closely linked to the peak period of the dominant wave components, suggesting that long‐wave modulation of wave breaking is a predominant mechanism controlling the intermittency of wave breaking across scales. Plain Language Summary: In the open ocean, wind waves of similar wavelength, period and direction combine to form wave groups, also known as sets to surfers and other beachgoers. In deep water, the individual waves in groups travel twice as fast as the groups themselves, and momentarily grow in height and steepness in the wave group center, making them theoretically more likely to break. We show, using field observations of wave breaking taken with digital video cameras, that waves in deep water are up to five times more likely to break in large wave groups than during lulls in the wave field. We also show that the large, dominant wave groups regularly initiate wave breaking at a wide range of scales. Our findings can be used to produce more accurate predictions of when individual waves will break, an active and relatively poorly understood area of current wave research. Key Points: Wave breaking activity and extent is significantly enhanced in wave groupsThe periodicity of the oceanic whitecap coverage is closely linked to the periodicity of dominant wave componentsThe intermittency characteristics of whitecap coverage approximately follow similarity scaling laws of other general stochastic processes [ABSTRACT FROM AUTHOR]

This work proposes the design, development and implementation of an artificial vision system based on stereo video, which is executed in an embedded system, to identify humanoid robots within a semi-structured area. The embedded system uses an Intel RealSense camera that, in addition to being able to obtain distances to objects due to its stereo vision, is capable of discriminating information after a distance threshold determined by the user, eliminating objects in the background of the scene and, through Image recognition based on a convolutional neural network recognizes the humanoid robots within it. The application of the system is based on the Robocup Humanoid League contest, where two teams of robots play soccer, so in addition to recognizing humanoid robots at different angles and distances, the system can classify recognized robots as companions or opponents (depending on the recognition of color marks they carry), emulating that, in the future, the proposed system will be mounted on another humanoid robot. [ABSTRACT FROM AUTHOR]

Stereo video has been widely applied in various video systems in recent years. Therefore, objective stereo video quality metric (SVQM) is highly necessary for improving the watching experience. However, due to the high dimensional data in stereo video, existing metrics have some defects in accuracy and robustness. Based on the characteristics of stereo video, this paper considers the coexistence and interaction of multi-dimensional information in stereo video and proposes an SVQM based on multi-dimensional analysis (MDA-SVQM). Specifically, a temporal-view joint decomposition (TVJD) model is established by analyzing and comparing correlation in different dimensions and adaptively decomposes stereo group of frames (sGoF) into different subbands. Then, according to the generation mechanism and physical meaning of each subband, histogram-based and LOID-based features are extracted for high and low frequency subband, respectively, and sGoF quality is obtained by regression. Finally, the weight of each sGoF is calculated by spatial-temporal energy weighting (STEW) model, and final stereo video quality is obtained by weighted summation of all sGoF qualities. Experiments on two stereo video databases demonstrate that TVJD and STEW adopted in MDA-SVQM are convincible, and the overall performance of MDA-SVQM is better than several existing SVQMs.

Electronic monitoring of fishery activities has drawn increasing attention. Deformable objects, noise from the wild sea surface, and dynamic background, however, make conventional tracking and segmentation methods unreliable. In this paper, we present an online 3D tracking and segmentation system for stereo video-based monitoring of rail fish catching on the wild sea surface. Based on the result of a pre-trained image object (fish) detector, a Kalman filtering-based tracking system overcomes the issues of low detection scores of deformed objects and of unreliable bounding boxes by rescoring multiple object proposals using spatial information in 3D. A clustering-and-scoring strategy is then applied on the depth map so that a plane classification method can effectively segment the objects from the dynamic background without any prior modeling. The object segmentation is further refined using fully connected conditional random fields based on color and geometric features. Using the segmentation results, we can measure the 3D lengths of objects and update the positions of bounding boxes to help tracking. Experimental results show that a reliable tracking and measurement performance under noisy and dynamic sea surface environment can be achieved. [ABSTRACT FROM AUTHOR]

There has been rapid uptake of stereo‐video‐based sampling techniques to collect species, abundance and body‐size information on fish assemblages and their associated habitats. Stereo‐video methods provide highly accurate estimates of body‐size, range and sampling area as well as a permanent record that can be cross‐checked or resampled for additional data. Due to these advantages, diver‐operated stereo‐video (stereo‐DOV) is becoming an increasingly popular alternative to underwater visual census.We provide a comprehensive guide for researchers using stereo‐DOVs to survey fish assemblages and their associated habitat. Information on stereo‐DOV design, video camera settings, field operations and video analysis are outlined.Stereo–DOV surveys permit rapid and simultaneous collection of data on fish diversity, abundance, length and behaviour as well as their associated habitat. However, biases associated with diver presence in the water and variation in detectability should be considered and are dependent on the location and focal species of the survey.We recommend using stereo‐DOVs for diver‐based surveys of non‐cryptic fish assemblages, given the advantages described. An increased uptake of this methodology, following the standard procedures described herein, will reduce variation in methodology, assist in the synthesis of data on continental and global scales and provide accurate information to improve fisheries management and conservation. [ABSTRACT FROM AUTHOR]

Transporting a stereo video stream over an unreliable channel can degrade severely the video quality. Network congestion is one of the important factors affecting the video quality. To improve the stereo video quality, a stereo video congestion control method, based on the macroblock importance, is proposed. Firstly, the importance of every macroblock is calculated according to its influence on the current frame and future frames. Then, combined with the importance information, all macroblocks of a frame are divided into three levels. Finally, the packets are transported with the help of RTP/RTCP protocol. We discard some not so important packets actively according to the importance levels when a congestion occurs. The results from NS2 simulation platform show that the stereo video quality is improved with the method proposed.

Typical oceanographic instruments are expensive, complex to build, and hard to deploy and require constant and specialized maintenance. In this paper, we present a cheap and simple technique to estimate a three-dimensional surface elevation map, η(x,y,t), the directional spectrum, and the main sea state parameters using inexpensive smartphones. The proposed methodology uses Time Lagged Cross Correlation (TLCC) between the audio signals from two independent video records to perform the frame synchronization. This makes the system much easier to deploy, where the main requirement is a fixed or moving platform close to the sea. The time records are mostly limited by the equipment storage space and battery life, although it can be easily replaced or recharged. Here, we pose the basis for an inexpensive yet powerful stereo reconstruction device and discuss its capabilities and limitations. The smartphone system capabilities were illustrated here by near shore experiment, at Leme beach in the Southeast of Brazil, and the results were compared against a pressure sensor. For this particular setup, the root mean square error in terms of significant wave height is of the order of 11% with perfect estimation of the peak period. The results are promising and demonstrate the validity and applicability of the technique.

In this letter, we present a full-reference (FR) quality assessment algorithm to assess the perceptual quality of natural stereoscopic three-dimensional (S3-D) videos. Toward the end, we rely on an empirical model for the joint statistics of motion and depth subband coefficients of an S3-D video frame. Specifically, we use the recently proposed bivariate generalized Gaussian distribution (BGGD) model for the joint statistics. In this letter, we show that the coherence of the covariance matrix of the BGGD varies in proportion with the perceptual video quality. We compute the coherence scores from the eigenvalues of the covariance matrix to estimate the amount of directional dependency between the motion and depth components. To estimate the overall spatial quality score, we apply off-the-shelf 2-D FR image quality assessment metrics on a frame-by-frame basis on both the views and average the frame wise scores. Finally, we pool the coherence and spatial quality scores to derive the overall quality for the S3-D video. An evaluation of the proposed algorithm on the IRCCYN, WaterlooIVC Phase I, and LFOVIA S3D video databases demonstrates its robust performance. The proposed algorithm is called depth- and motion-based 3-D video quality evaluator. [ABSTRACT FROM AUTHOR]

A new bit-allocation and rate-control algorithm for stereo video coding was proposed to improve its coding efficiency and stream reliability.The conventional quadric rate-distortion model was improved and a new one was introduced by incorporating the human visual system（HVS） characteristics.Experimental results show that the improved new rate-distortion model is more precise than the conventional one and the proposed new algorithm can efficiently implement bit allocation and rate control according to coding parameters.

The efficient search and retrieval of the increasing volume of stereo videos drives the need for the semantic description of its content. The analysis and description of the disparity (depth) data available on such videos, offers extra information, either for developing better video content search algorithms, or for improving the 3D viewing experience. Taking the above into account, the purpose of this paper is twofold. First, to provide a mathematical analysis of the relation of object motion between world and display space and on how disparity changes affect the 3D viewing experience. Second, to propose algorithms for semantically characterizing the motion of an object or object ensembles along any of the X, Y, Z axis. Experimental results of the proposed algorithms for semantic motion description in stereo video content are given. [ABSTRACT FROM AUTHOR]

We describe a method for tracking the path of animals in the field, based on stereo videography and aiming-angle measurements, combined in a single, rotational device. In open environments, this technique has the potential to extract multiple 3D positions per second, with a spatial uncertainty of <1 m (rms) within 300 m of the observer, and <0.1 m (rms) within 100 m of the observer, in all directions. The tracking device is transportable and operated by a single observer, and does not involve any animal tagging. As a video of the moving animal is recorded, track data can easily be completed with behavioural data. We present a prototype device based on accessible components that achieves about 70% of the theoretical maximal range. We show examples of bird ground and flight tracks, and discuss the strengths and limits of the method, compared with existing fine-scale (e.g. fixed-camera stereo videography) and largescale tracking methods (e.g. GPS tracking). [ABSTRACT FROM AUTHOR]

We present a stereo algorithm that is capable of estimating scene depth information with high accuracy and in real time. The key idea is to employ an adaptive cost-volume filtering stage in a dynamic programming optimization framework. The per-pixel matching costs are aggregated via a separable implementation of the bilateral filtering technique. Our separable approximation offers comparable edge-preserving filtering capability and leads to a significant reduction in computational complexity compared to the traditional 2D filter. This cost aggregation step resolves the disparity inconsistency between scanlines, which are the typical problem for conventional dynamic programming based stereo approaches. Our algorithm is driven by two design goals: real-time performance and high accuracy depth estimation. For computational efficiency, we utilize the vector processing capability and parallelism in commodity graphics hardware to speed up this aggregation process over two orders of magnitude. Over 90 million disparity evaluations per second [the number of disparity evaluations per seconds (MDE/s) corresponds to the product of the number of pixels and the disparity range and the obtained frame rate and, therefore, captures the performance of a stereo algorithm in a single number] are achieved in our current implementation. In terms of quality, quantitative evaluation using data sets with ground truth disparities shows that our approach is one of the state-of-the-art real-time stereo algorithms. [ABSTRACT FROM AUTHOR]

This paper presents a new implementation of a high-definition image-processing engine, which mainly targets the 3-dimensional (3D) visualization and stereo video stream display of binocular display equipment. The engine is compatible with the mainstream analog and digital stereo videos in component format and is able to receive stereo composite video broadcast signals using an integrated analog stereo video decoder. The four modules include a spatiotemporal scaling transform engine, a 2D-3D converter, an image animating engine, and a 2D scalar operating in pipeline architecture to implement the video format conversion and the stereo effect enhancement. Furthermore, the data access, hardware structure, and system-level configurations are optimized. Finally, the proposed architecture is realized by 0.18 μm CMOS technology. The application-specific integrated circuit verification results show that the engine can generate a strong feeling of 3D immersion and high-definition image quality with minimal flicker. The chip has wide compatibility and an uppermost 1080P-processing capacity, which has approximately 3.5 million gates with about 43 mm die size. [ABSTRACT FROM AUTHOR]

The depth information of the scene indicates the distance between the object and the camera, and depth extraction is a key technology in 3D video system. The emergence of Kinect makes the high resolution depth map capturing possible. However, the depth map captured by Kinect can not be directly used due to the existing holes and noises, which needs to be repaired. We propose a texture combined inpainting algorithm in this paper. Firstly, the foreground is segmented combined with the color characteristics of the texture image to repair the foreground of the depth map. Secondly, region growing is used to determine the match region of the hole in the depth map, and to accurately position the match region according to the texture information. Then the match region is weighted to fill the hole. Finally, a Gaussian filter is used to remove the noise in the depth map. Experimental results show that the proposed method can effectively repair the holes existing in the original depth map and get an accurate and smooth depth map, which can be used to render a virtual image with good quality. [ABSTRACT FROM AUTHOR]

The important requirements for stereo video retargeting are threefold: keeping temporal coherence, preventing depth distortion, and minimizing shape distortions of the retargeted video. To meet these requirements, the left and right video sequences are divided into groups of frames (GoFs), where the GoF is a basic unit for the seam carving and we assign a set of fixed seams for all frames within the GoF. To determine the fixed seams for each GoF, we need to find the GoF boundary in the video first. Then, the representative frame for each GoF is generated by considering the spatial saliency and temporal coherence. Also, the confidence of the stereoscopic correspondence between the left and right frames is considered to prevent depth distortion. [ABSTRACT FROM AUTHOR]

This paper surveys major techniques in 3D communications area, which covers the whole pipeline of the 3D video communication framework, including 3D content creation, data representation, compression, delivery, decompression, post-processing, and 3D scene rendering stages. Both the current state-of-the-art, stereo 3D, and future trend, free-viewpoint 3D, are demonstrated in details. On the other hand, the paper highlights a few features in the emerging 4G wireless systems that are critical for 3D communications system design. At the end, the topics with potential but challenges, for example 3D over 4G networks, distributed 3D video coding, 3D multi-user communication, scalability and universal 3D access, are discussed and pointed out to audiences for further investigation. Copyright © 2010 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Abstract: This paper gives an end-to-end overview of 3D video and free viewpoint video, which can be regarded as advanced functionalities that expand the capabilities of a 2D video. Free viewpoint video can be understood as the functionality to freely navigate within real world visual scenes, as it is known for instance from virtual worlds in computer graphics. 3D video shall be understood as the functionality that provides the user with a 3D depth impression of the observed scene, which is also known as stereo video. In that sense as functionalities, 3D video and free viewpoint video are not mutually exclusive but can very well be combined in a single system. Research in this area combines computer graphics, computer vision and visual communications. It spans the whole media processing chain from capture to display and the design of systems has to take all parts into account, which is outlined in different sections of this paper giving an end-to-end view and mapping of this broad area. The conclusion is that the necessary technology including standard media formats for 3D video and free viewpoint video is available or will be available in the future, and that there is a clear demand from industry and user for such advanced types of visual media. As a consequence we are witnessing these days how such technology enters our everyday life [Copyright &y& Elsevier]

It is well known that the human visual system can perceive high frequencies in 3D stereo video, even if that information is present in only one of the views. Therefore, the best perceived 3D stereo video quality may be achieved by asymmetric coding where the reference and auxiliary (right and left) views are coded at unequal PSNR. However, the questions of what is the best level of asymmetry in order to maximize the perceived quality and whether asymmetry should be achieved by spatial resolution reduction or PSNR (quality) reduction have been open issues. We conducted extensive subjective tests, which indicate that if the reference view is encoded at sufficiently high quality and the auxiliary view is encoded at a lower quality but above a certain PSNR threshold, then the degradation in 3D video quality is unnoticeable. Since asymmetric coding by PSNR reduction gives finer control over achievable PSNR values over spatial resolution reduction, asymmetry by PSNR reduction allows us to encode at a point more close to this just-noticeable asymmetry PSNR threshold; hence will be preferred over the spatial resolution reduction method. Subjective tests also indicate that below this just-noticeable asymmetry threshold, where subtle artifacts start to appear, symmetric coding performs better than asymmetric coding in terms of perceived 3D video quality. Therefore, we show that the choice between asymmetric vs. symmetric coding depends on PSNR; hence, the available total bitrate. This paper also proposes a novel asymmetric scalable stereo video coding framework to enable adaptive stereoscopic video streaming taking full advantage of these observations and subjective test results. [ABSTRACT FROM AUTHOR]

The article compares the relative abundance and size of redthroat emperor Lethrinus miniatus in areas open and closed to fishing at the Houtman Abrolhos Islands in Western Australia. The change in the species population during study period is assessed. Information on the geography and fishing areas of the Houtman Abrolhos Islands is presented. The method and materials used to collect samples are detailed. The relative abundance, length, and behavior of Lethrinus miniatus in open and closed fishing areas are assessed.

It is important to reduce data redundancy of stereo video in practical applications. In this paper, first, a data embedding method for stereo video (DEMSV) is investigated by embedding the encoding data into the reference frame to encode stereo video. It can use only one channel to transfer all the video data and the receiver can choose a monocular video decoder or stereo video decoder adaptively. Then, introducing the joint prediction scheme in the coding process of DEMSV, we propose a novel data embedding method for H.264 stereo video codec with joint prediction scheme (DEMSV-JPS) to achieve high coding efficiency. Experimental results show that the proposed method can obtain high peak signal-to-noise ratio (PSNR) and compression ratio (at least 33 dB for the test sequence). Comparing the testing methods using JPS and without using JPS, we prove that JPS can further improve the objective and visual quality. DEMSV-JPS shows such advantages and will be suitable to applications in real-time environments of stereo-video transmission. [ABSTRACT FROM AUTHOR]

Sub-project 6 of the European Integrated Project APROSYS, implemented within the 6th Framework Programme, has developed an integrated safety system for side impact protection. For this, two different innovative technologies have been applied for the first time as a technology showcase in a car: shape memory alloy based actuators and a pre-crash sensing subsystem using radar and stereo-vision observing the side of a vehicle. Several technical challenges had to be met during development work. The sensing subsystem detects an imminent collision and decides about the probability of a collision. It activates the actuator before the crash occurs in order to reduce intrusion into the passenger compartment. This article describes the sensing subsystem, the actuator subsystem and finally the testing programme that was executed to evaluate the complete system. [ABSTRACT FROM AUTHOR]