Search

The plenoptic function is commonly used to model image-based rendering (IBR) in terms of sampling and reconstruction. In this paper, we study the influence of scene geometry on the spectrum of a plenoptic function using spectral analysis. As the scene geometry becomes more complicated, the spectrum of the plenoptic function will broaden. Therefore, the bandwidth of the plenoptic function is closely related to the scene geometry, and the end goal of the bandwidth is to determine the minimum sampling rate (i.e., the minimum number of cameras) for IBR. Under the spectral analysis, we mathematically derive an analytical function to determine the spectral support of the plenoptic function. Based on the support of the plenoptic spectrum, the minimum number of cameras is determined to reconstruct the plenoptic function up to a certain frequency that allows a certain reconstruction quality when performing IBR. Furthermore, we derive a reconstruction filter for the plenoptic sampling for irregularly shaped scenes. Last, the experimental results are presented and compared with the theory. [ABSTRACT FROM AUTHOR]

We present an occlusion-aware unsupervised neural network for jointly learning three low-level vision tasks from monocular videos: depth, optical flow, and camera motion. The system consists of three different predicting sub-networks simultaneously coupled by combined loss terms and is capable of computing each task independently on test samples. Geometric constraints extracted from scene geometry which have traditionally been used in bundle adjustment or pose-graph optimization are formed as various self-supervisory signals during our end-to-end learning approach. Different from prior works, our image reconstruction loss also takes account of optical flow. Moreover, we impose novel 3D flow consistency constraints over the predictions of all the three tasks. By explicitly modeling occlusion and taking utilization of both 2D and 3D geometry relationships, abundant geometric constraints are formed over estimated outputs, enabling the system to capture both low-level representations and high-level cues to infer thinner scene structures. Empirical evaluation on the KITTI dataset demonstrates the effectiveness and improvement of our approach: (1) monocular depth estimation outperforms state-of-the-art unsupervised methods and is comparable to stereo supervised ones; (2) optical flow prediction ranks top among prior works and even beats supervised and traditional ones especially in non-occluded regions; (3) pose estimation outperforms established SLAM systems under comparable input settings with a reasonable margin.

Nowadays, touch screens are widely used in many commercial devices. In this paper, we present an inexpensive Webcam-based touch screen system based on a simple and generic finger detection technique. Only a single Webcam and a regular screen are used to achieve this goal. By using the scene geometry and the camera model, we can determine the position of the finger on the screen. The technique we propose is simple and works well under various surrounding environments while the detecting method suits many finger colors and shapes. Our approach can also be used for realtime applications. Experimental results and practical applications are provided to ascertain the proposed method. [ABSTRACT FROM PUBLISHER]

Given a set of high-resolution images of a scene, it is often desirable to predict the scene's appearance from viewpoints not present in the original data for purposes of change detection. When significant 3-D relief is present, a model of the scene geometry is necessary for accurate prediction to determine surface visibility relationships. In the absence of an a priori high-resolution model (such as those provided by LIDAR), scene geometry can be estimated from the imagery itself. These estimates, however, cannot, in general, be exact due to uncertainties and ambiguities present in image data. For this reason, probabilistic scene models and reconstruction algorithms are ideal due to their inherent ability to predict scene appearance while taking into account such uncertainties and ambiguities. Unfortunately, existing data structures used for probabilistic reconstruction do not scale well to large and complex scenes, primarily due to their dependence on large 3-D voxel arrays. The work presented in this paper generalizes previous probabilistic 3-D models in such a way that multiple orders of magnitude savings in storage are possible, making high-resolution change detection of large-scale scenes from high-resolution aerial and satellite imagery possible. Specifically, the inherent dependence on a discrete array of uniformly sized voxels is removed through the derivation of a probabilistic model which represents uncertain geometry as a density field, allowing implementations to efficiently sample the volume in a nonuniform fashion. [ABSTRACT FROM AUTHOR]

Reconstruction of 3D scene geometry is an important element for scene understanding, autonomous vehicle and robot navigation, image retrieval, and 3D television. We propose accounting for the inherent structure of the visual world when trying to solve the scene reconstruction problem. Consequently, we identify geometric scene categorization as the first step toward robust and efficient depth estimation from single images. We introduce 15 typical 3D scene geometries called stages, each with a unique depth profile, which roughly correspond to a large majority of broadcast video frames. Stage information serves as a first approximation of global depth, narrowing down the search space in depth estimation and object localization. We propose different sets of low-level features for depth estimation, and perform stage classification on two diverse data sets of television broadcasts. Classification results demonstrate that stages can often be efficiently learned from low-dimensional image representations. [ABSTRACT FROM AUTHOR]

Making robots know people's place concepts has attracted researchers for decades. People believe that this capability will firmly benefit not only robot–human interaction but also reasonable and social robot behaviors, or even traditional problems in robot research such as object detection. Focusing on place classification, this paper builds a kind of native pure 3D geometric description to capture place layouts based on common point clouds. This perspective enables our method to naturally accommodate various illuminations, including extremely bad lighting for which traditional image methods cannot work properly. The space of a place is first divided into 3D voxels. The cardinal orientations of this space are then extracted, and the geometric attributes of the voxels are subsequently represented based on the cardinal orientations. The voxels with geometric attributes are defined as the cardinal-direction prototyping blocks (CDPBs). Next, the CDPB distribution for a scene is calculated by qualitative spatial description technology, thereby obtaining the complete place description. Given the sparse description, the sparse random forest (SRF) is used for learning. The experiments indicate that the CDPB-based method outperforms the current 3D geometric method and its mixed method, and it has good time performance. The main advantages of our method are that it does not require any strict hypotheses on surfaces, such as planar surfaces, it requires smaller fusion windows to attain satisfactory classification rates, it can be used in extreme lighting environments, and its parameter selection is easy. • A native place–layout representation method of an appropriate granularity. • A novel qualitative description of point clouds. • A pure 3D method that can be used in extreme lighting environments. • Outperforms 3D geometry mixed method and has better real-time property. • We attempt to discover the limitations of the pure 3D geometry method. [ABSTRACT FROM AUTHOR]

We present an occlusion-aware unsupervised neural network for jointly learning three low-level vision tasks from monocular videos: depth, optical flow, and camera motion. The system consists of three different predicting sub-networks simultaneously coupled by combined loss terms and is capable of computing each task independently on test samples. Geometric constraints extracted from scene geometry which have traditionally been used in bundle adjustment or pose-graph optimization are formed as various self-supervisory signals during our end-to-end learning approach. Different from prior works, our image reconstruction loss also takes account of optical flow. Moreover, we impose novel 3D flow consistency constraints over the predictions of all the three tasks. By explicitly modeling occlusion and taking utilization of both 2D and 3D geometry relationships, abundant geometric constraints are formed over estimated outputs, enabling the system to capture both low-level representations and high-level cues to infer thinner scene structures. Empirical evaluation on the KITTI dataset demonstrates the effectiveness and improvement of our approach: (1) monocular depth estimation outperforms state-of-the-art unsupervised methods and is comparable to stereo supervised ones; (2) optical flow prediction ranks top among prior works and even beats supervised and traditional ones especially in non-occluded regions; (3) pose estimation outperforms established SLAM systems under comparable input settings with a reasonable margin. [ABSTRACT FROM AUTHOR]