Search

Purpose There are conflicting reports as to whether there is a binocular advantage or disadvantage when reading with central vision loss. This study examined binocular reading summation in patients with macular degeneration. Methods Seventy‐one patients with bilateral central vision loss [mean age: 63 (S.D. = 21) years] participated. Reading performances during binocular and monocular viewing with the better eye (i.e., the eye with the best monocular visual acuity) were evaluated using different versions of the Italian MNREAD reading chart (www.precision‐vision.com). Fixation stability and preferred retinal loci (PRLs) were recorded monocularly for each eye. The overall sample was split into inhibition, equality, and summation groups based on the binocular ratio (i.e., binocular/monocular) of the maximum reading speed. Results 41% of patients experienced binocular inhibition, 42% summation, and 17% equality. Binocular reading speed of the inhibition group was approximately 30 words per minute slower than those of the equality and summation groups, although the inhibition group had the best visual acuity. These patients generally had monocular PRLs in non‐corresponding locations temporal or nasal to the scotoma, had the largest interocular acuity difference and lacked residual stereopsis. The three groups did not differ in fixational control, contrast sensitivity or critical print size. Conclusions Equal proportions of patients with central vision loss show binocular reading summation and inhibition. Patients with binocular reading inhibition have poorer reading performance and different clinical characteristics than those with binocular reading summation and equality.

No-reference stereoscopic image quality assessment (NR SIQA) has aroused much attention in the multimedia application, but most current works cannot achieve reliable accuracy due to the non-visual meaning image patches and the weak biological visual models. Inspired by this, we propose a novel NR SIQA model by the introduction of the meaningful superpixel visual patches and the significant binocular perception models. Following the visual perceptual posterior probability theory, local monocular superpixel spatial entropy and the natural scene statistics (NSS) features are firstly utilized to present the high level semantic perceptual mechanism and image naturalness. Then the binocular biological models are applied to extract the binocular features to simulate the inner binocular interaction information, which well complement with the monocular features. Finally, a support vector regression (SVR) model is utilized to predict the objective quality score. Experiments are conducted on four popular stereoscopic image Databases, and the results demonstrate that the proposed model obtains a good consistency with the subjective scores, and has more reliable performance than previous works in terms of prediction accuracy and generalization capability.

The most prominent characteristic of a stair is that it has rigid form with periodic pattern of its steps. In this work, we exploit this periodic characteristic in view of geometrical rules. As a stair consists of equidistant nosing lines, under a perspective projection of camera, the projection of these lines on an image follows a projective chirplet transform. We propose to detect a stair by finding a group of lines that best satisfies a projective chirp model. The most advantage of the proposed techniques is that some missed noising lines and thus whole stair could be recovered. We validate the proposed method on both artificial and real datasets. The experimental results show a higher detection rate on different datasets. Finally, a real application alarming the visually impaired about stairs in indoor environments has been conducted and obtained 88.37% of accuracy. The implementations and datasets are made publicly available.

In this paper, we propose an unsupervised learning framework to address the problems of the inaccurate inference of depth details and the loss of spatial information for monocular depth estimation. First, as an unsupervised technique, the proposed framework takes easily collected stereo image pairs instead of ground truth depth data as inputs for training. Second, we design a rectangle convolution to capture global dependencies between neighboring pixels across entire rows or columns in an image, which can bring significant promotion on depth details inference. Third, we propose a learned depth refinement module including a color-guided refinement layer and a learned composite proximal operator to preserve depth discontinuities and obtain high quality depth map. The proposed network is fully differentiable and end-to-end trainable. Extensive experiments evaluated on KITTI, Cityscapes and Make3D dataset demonstrate our state-of-the-art performance and good cross-dataset generalization ability.

Purpose To evaluate the performance and safety of the Extended Depth of Focus Implantable Collamer® Lens (EDOF ICL) for improvement of uncorrected near, intermediate and distance visual acuity in phakic subjects with myopia and presbyopia. Design Prospective multicenter study. Methods Presbyopic subjects who required an EDOF ICL in the range of -0.50 D to -18.00 D, exhibited ≤ 0.75 D refractive astigmatism and required from +1.00 to +2.50 D reading add were implanted bilaterally. Assessments at 6 months included uncorrected near, intermediate and distance visual acuities, defocus curves, contrast sensitivity, responses to the National Eye Institute Refractive Error Quality of Life Questionnaire and a Task Assessment Questionnaire. Results A total of 34 subjects completed the study. Investigators targeted emmetropia in all eyes. Mean binocular uncorrected near, intermediate and distance visual acuities measured logMAR -0.01 ± 0.05 (20/20), -0.02 ± 0.08 (20/19) and 0.07 ± 0.10 (20/23), respectively. Mean monocular uncorrected near, intermediate and distance visual acuities measured logMAR 0.068 ± 0.09 (20/23), 0.062 ± 0.10 (20/23) and 0.16 ± 0.12 (20/29). There were no clinically or statistically significant differences in contrast sensitivity between baseline and 6 months under any testing conditions. Subjects reported significant improvements in measures of vision-related quality of life and ability to perform tasks at all distances without glasses or contact lenses. Overall, satisfaction with the EDOF ICL was high: postoperatively, 91.2% of subjects were satisfied with their vision. Conclusion This multicenter, prospective clinical investigation demonstrated the ability of the EDOF ICL to correct myopia and presbyopia, resulting in improvement of uncorrected near, intermediate and distance visual acuity without compromising the quality of vision. The EDOF ICL allowed subjects to perform tasks of daily living without glasses or contact lenses. Subjects reported significant improvements in quality of life with high levels of spectacle independence and satisfaction.

Vision is an essential element for maintaining the balance in the standing position or for walking. The purpose of this study was to investigate the effects of dominant or non-dominant eye blocking on gait and balance. In 17 healthy individuals, static balance and gait function were measured when the dominant or non-dominant eye were blocked. Static balance was measured by the Wii balance board and the Balancia software program. We also measured gait function (spatio-temporal variables) and foot lateral deviations. In the present results, the path length, path velocity, and path area were significantly decreased during blocking dominant on static balance function (p

Localization, navigation, and mapping using vision-based algorithms are an active topic in underwater robotic applications. Although many algorithms developed in recent years, especially in the ground and areal robotic communities, directly applying those methods in underwater navigation remain challenging due to the visual degradation induced by the medium. In this paper, we proposed UW-SLAM (Underwater SLAM), a new monocular visual SLAM algorithm focused on the underwater environment which addresses the turbidity and dynamism. The proposed method was evaluated with several underwater datasets with comparison to the state of the art monocular SLAM methods.

Background Given the widely distributed network of midbrain, pontine, cerebellar, and cortical areas involved in the neural control of vergence, one might expect various vergence deficits in stroke patients. In this article, we investigated the localizing value of bedside vergence testing with respect to different supratentorial and infratentorial infarction locations. Methods Three hundred five stroke patients and 50 age-matched controls were examined prospectively by means of bedside tests to assess slow and fast binocular (i.e., symmetrical) as well as slow and fast monocular (i.e., asymmetrical) convergence. Infarction locations, as identified on MRI, were correlated with vergence performance using multinomial logistic regression. Results Vergence deteriorated with age in both stroke patients and healthy controls. Most infarction locations did not show significant associations with vergence parameters, apart from cases with parietal lobe lesions, which exhibited insufficient asymmetrical, slow and fast vergence for both the left and the right eye. Finally, patients with severe ischemic small vessel disease showed a slight but significant decrease in their fast binocular vergence performance. Conclusions There is only a limited localizing value of vergence deficits in stroke. Parietal lobe infarctions are more frequently associated with insufficient binocular and monocular vergence. Midbrain strokes were too few to draw final conclusions. However the most robust factor to emerge from our data is age. Older subjects show poor slow binocular as well as slow and fast monocular vergence. Extended white matter lesions are also correlated with deficient vergence ability suggesting a role for subcortical wide range connections in maintaining an intact vergence circuitry.

Background Visual acuity has been a significant outcome measure in clinical trials for patients suffering from neuro-ophthalmological diseases and multiple sclerosis; however, there are limited data on the comparison of various testing strategies in pediatric patients with these disorders. Clinical trials using vision as an outcome could include a variety of tools to assess the acuity, including 2-m and 4-m standardized retroilluminated charts. Methods We investigated the difference in Early Treatment Diabetic Retinopathy Study (ETDRS) scores obtained using 2-m and 4-m charts, as well as the impact of optic neuritis, use of vision correction, age, and gender on visual acuity data from 71 patients with pediatric neuroimmunological conditions in a cross-sectional study. Results We determine that the ETDRS letter scores obtained using 4-m charts are on average 3.43 points less (P = 0.0034) when testing monocular ETDRS letter scores and on average 4.14 points less (P = 0.0008) when testing binocular ETDRS letter scores, relative to that obtained using the 2-m charts. However, we find that when performing monocular testing, optic neuritis in the eye being tested did not result in a statistically significant difference between 2-m and 4-m ETDRS letter scores. Conclusions Although visual acuity charts are formatted by the distance, there are significant differences in the number of letters correctly identified between 2-m and 4-m charts. Although the differences may not impact the clinical acuity, research protocols should consider these differences before collapsing data across disparate studies.

BACKGROUND: Registration as sight impaired allows access to services important for patients. The rates of sight impairment due to visual field loss are underestimated. Previous work has shown that evaluation of visual field defects in both eyes produces poor agreement among ophthalmologists for categorisation of patients as eligible for sight impairment registration. AIM: To evaluate the impact of binocular summation of both eye glaucomatous visual field defects on agreement for sight impairment registration. METHODS: Thirty consultant ophthalmologists (Graders), graded 50 glaucomatous visual field sets. Each consisted of both monocular fields and summated binocular plots. Graders classified the visual field sets as sight impaired (SI), severely sight impaired (SSI) or neither. Trichotomous, (SI, SSI or no sight impairment) and dichotomous (any sight impairment versus no sight impairment) concordance values were estimated for the group of graders as a whole and for glaucoma and non-glaucoma experts. RESULTS: For trichotomous analysis the overall kappa agreement rate was 0.29; for dichotomous analysis it was 0.40. There was no material difference between glaucoma experts and non-experts. CONCLUSION: Overall agreement was modest. Grading for SI showed the poorest levels of agreement. Using binocular fields does not appear to improve concordance for sight impairment registration. Moreover, there is no difference in agreement between glaucoma and non-glaucoma experts. An overall score for visual disability using mean deviation may be a more pragmatic approach.

In the field of robotics, a major challenge is achieving high levels of autonomy with small vehicles that have limited mass and power budgets. The main motivation for designing such small vehicles is that compared to their larger counterparts, they have the potential to be safer, and hence be available and work together in large numbers. One of the key components in micro robotics is efficient software design to optimally utilize the computing power available. This paper describes the computer vision and control algorithms used to achieve autonomous flight with the [Formula: see text]30[Formula: see text]g tailless flapping wing robot, used to participate in the International Micro Air Vehicle Conference and Competition (IMAV 2018) indoor microair vehicle competition. Several tasks are discussed: line following, circular gate detection and fly through. The emphasis throughout this paper is on augmenting traditional techniques with the goal to make these methods work with limited computing power while obtaining robust behavior.

This work proposes an approach of image-to-image translation deep learning model called cycle consistent adversarial networks for reconstructing the digital surface model from monocular aerial imagery. The proposed model architecture consists of generators with the encoder–decoder system with skip connection and two discriminators that penalize structures at the scale of patches. The objective function of the cycleGAN has improved by adding L1 loss for training on paired samples. Conditional GAN is used as a baseline model in this study. The proposed approach showed higher reconstruction capabilities for generating a surface model from aerial imagery than previous studies that used conditional GAN. The proposed architecture exhibited a strong potential in reconstructing a surface model from single aerial imagery with the capacity to generalize multiple cities and built-up environments. The results can be useful in urban studies and visualization of urban data for better governance.

In order to quick response to the rapid changes of mobile platforms in complex situations such as speedy changing direction or camera shake, visual odometry/visual simultaneous localization and mapping (VO/VSLAM) always needs a high frame rate vision sensor. However, the high frame rate of the sensor will affect the real-time performance of the odometry. Therefore, we need to investigate how to make a balance between the frame rate and the pose quality of the sensor. In this paper, we propose an automatic key-frame method based on the improved PWC-Net for mobile platforms, which can improve the pose tracking quality of odometry, the error caused by dynamic blur and the global robustness. First, a two-step decomposition is used to calculate the change of inter-frame attitude, and then, key-frames are added by the improved PWC-Net or automatically selected based on the motion state of the vehicle predicted by pose change with a short time interval. To evaluate the method, we conduct extensive experiments on KITTI dataset based on monocular visual odometry. The results indicate that our method can keep the pose tracking quality while ensuring the real-time performance.

Purpose: To determine whether monocularly- and binocularly-induced spherical and meridional blur and aniseikonia had similar effects on stereopsis thresholds. Methods: Twelve participants with normal binocular vision viewed McGill modified random dot stereograms to determine stereoacuities in a four-alternative forced-choice procedure. Astigmatism was induced by placing trial lenses in front of the eyes. Twenty-three conditions were used, consisting of zero (no lens), +1 D and +2 D spheres and cylinders at axes 180, 45 and 90 in front of the right eye, and the following binocular combinations of both lens powers: R × 180/L × 180, R × 45/L × 45, R × 90/L × 90, R sphere/L sphere, R × 180/L × 90, R × 45/L × 135, R × 90/L × 180. Aniseikonia was induced by placing magnifying lenses in front of the eyes. Twenty-three conditions were used, consisting of zero, 6% and 12% overall magnification and both magnifications at axes 180, 45 and 90 in front of the right eye only, and the following binocular combinations using 3% and 6% lenses: R × 90/L × 90, R × 45/L × 45, R × 180/L × 180, R overall/L overall, R × 90/L × 180, R × 45/L × 135, and R × 180/L × 90. Results: Stereopsis losses for binocular blur effects with parallel axes (non-anisometropic) were the same as for monocular blur effects of the same axes, and these were strongly dependent on axis (spherical blur and ×90 had the greatest effects). Binocular blur effects with orthogonal axes had greater effects than with parallel axes, with the axis combination of the former having no effect (e.g. R × 90/L × 180 was similar to R × 45/L × 135). For induced aniseikonia, splitting the magnifications between the eyes improved stereopsis slightly, and the effects were not dependent on axis. Conclusion: Binocular blur affects stereopsis similarly to monocular meridional blur if axes in the two eyes are parallel, whereas the effect is greater if the axes are orthogonal. In meridional aniseikonia, splitting magnification between the right and left lenses produces a small improvement in stereopsis that is independent of axis direction and right/left combination.

The present natural preference study investigated infants 4 and 7 months of age for their ability to respond to phantom contoubrs, illusory surfaces generated by half-occlusions in a stereoscopic display consisting of a pair of parallel vertical lines. The left line in the half-image for the right eye and the right line in the half-image for the left eye have a gap in the middle. The visual system accounts for the binocular unmatched gaps by perceiving an illusory contour. Infants in the experimental condition were presented with a standard phantom stereogram displaying a phantom contour versus a non-standard phantom stereogram, the half-images of which were exchanged. This stereogram evokes the impression of two small separate illusory contours. In both stereograms, the gaps moved up and down. The participants aged 7 but not 4 months preferred looking at the standard phantom stereogram. A control condition supported the hypothesis that the infants 7 months of age in the experimental condition indeed responded to the coherent illusory surface instead of simply detecting differences in the geometric arrangement of the half-occlusions. The results hence indicate that infants are able to extract spatial information from monocular regions in a binocular display.

This paper proposes a fused 3D transparent visualization method with the aim of achieving see-through imaging of large-scale cultural heritage by combining photogrammetry point cloud data and 3D reconstructed models. 3D reconstructed models are efficiently reconstructed from a single monocular photo using deep learning. It is demonstrated that the proposed method can be widely applied, particularly to instances of incomplete cultural heritages. In this study, the proposed method is applied to a typical example, the Borobudur temple in Indonesia. The Borobudur temple possesses the most complete collection of Buddhist reliefs. However, some parts of the Borobudur reliefs have been hidden by stone walls and became not visible following the reinforcements during the Dutch rule. Today, only gray-scale monocular photos of those hidden parts are displayed in the Borobudur Museum. In this paper, the visible parts of the temple are first digitized into point cloud data by photogrammetry scanning. For the hidden parts, a 3D reconstruction method based on deep learning is proposed to reconstruct the invisible parts into point cloud data directly from single monocular photos from the museum. The proposed 3D reconstruction method achieves 95% accuracy of the reconstructed point cloud on average. With the point cloud data of both the visible parts and the hidden parts, the proposed transparent visualization method called the stochastic point-based rendering is applied to achieve a fused 3D transparent visualization of the valuable temple.

Depth is an essential component for various scene understanding tasks and for reconstructing the 3D geometry of the scene. Estimating depth from stereo images requires multiple views of the same scene to be captured which is often not possible when exploring new environments with a UAV. To overcome this monocular depth estimation has been a topic of interest with the recent advancements in computer vision and deep learning techniques. This research has been widely focused on indoor scenes or outdoor scenes captured at ground level. Single image depth estimation from aerial images has been limited due to additional complexities arising from increased camera distance, wider area coverage with lots of occlusions. A new aerial image dataset is prepared specifically for this purpose combining Unmanned Aerial Vehicles (UAV) images covering different regions, features and point of views. The single image depth estimation is based on image reconstruction techniques which uses stereo images for learning to estimate depth from single images. Among the various available models for ground-level single image depth estimation, two models, 1) a Convolutional Neural Network (CNN) and 2) a Generative Adversarial model (GAN) are used to learn depth from aerial images from UAVs. These models generate pixel-wise disparity images which could be converted into depth information. The generated disparity maps from these models are evaluated for its internal quality using various error metrics. The results show higher disparity ranges with smoother images generated by CNN model and sharper images with lesser disparity range generated by GAN model. The produced disparity images are converted to depth information and compared with point clouds obtained using Pix4D. It is found that the CNN model performs better than GAN and produces depth similar to that of Pix4D. This comparison helps in streamlining the efforts to produce depth from a single aerial image.

Predicting depth from a single image is an attractive research topic since it provides one more dimension of information to enable machines to better perceive the world. Recently, deep learning has emerged as an effective approach to monocular depth estimation. As obtaining labeled data is costly, there is a recent trend to move from supervised learning to unsupervised learning to obtain monocular depth. However, most unsupervised learning methods capable of achieving high depth prediction accuracy will require a deep network architecture which will be too heavy and complex to run on embedded devices with limited storage and memory spaces. To address this issue, we propose a new powerful network with a recurrent module to achieve the capability of a deep network while at the same time maintaining an extremely lightweight size for real-time high performance unsupervised monocular depth prediction from video sequences. Besides, a novel efficient upsample block is proposed to fuse the features from the associated encoder layer and recover the spatial size of features with the small number of model parameters. We validate the effectiveness of our approach via extensive experiments on the KITTI dataset. Our new model can run at a speed of about 110 frames per second (fps) on a single GPU, 37 fps on a single CPU, and 2 fps on a Raspberry Pi 3. Moreover, it achieves higher depth accuracy with nearly 33 times fewer model parameters than state-of-the-art models. To the best of our knowledge, this work is the first extremely lightweight neural network trained on monocular video sequences for real-time unsupervised monocular depth estimation, which opens up the possibility of implementing deep learning-based real-time unsupervised monocular depth prediction on low-cost embedded devices.

Near infrared bands (NIR) provide rich information for many remote sensing applications. In addition to deriving useful indices to delineate water and vegetation, near infrared channels could also be used to facilitate image pre-processing. However, synthesizing bands from RGB spectrum is not an easy task. The inter-correlations between bands are not clearly identified in physical models. Generative adversarial networks (GAN) have been used in many tasks such as generating photorealistic images, monocular depth estimation and Digital Surface Model (DSM) refinement etc. Conditional GAN is different in that it observes some data as a condition. In this paper, we explore a cGAN network structure to generate a NIR spectral band that is conditioned on the input RGB image. We test different discriminators and loss functions, and evaluate results using various metrics. The best simulated NIR channel has a mean absolute error of around 5 percent in Sentinel-2 dataset. In addition, the simulated NIR image can correctly distinguish between various classes of landcover.

Most existing algorithms for depth estimation from single monocular images need large quantities of metric groundtruth depths for supervised learning. We show that relative depth can be an informative cue for metric depth estimation and can be easily obtained from vast stereo videos. Acquiring metric depths from stereo videos is sometimes impracticable due to the absence of camera parameters. In this paper, we propose to improve the performance of metric depth estimation with relative depths collected from stereo movie videos using existing stereo matching algorithm. We introduce a new "Relative Depth in Stereo" (RDIS) dataset densely labelled with relative depths. We first pretrain a ResNet model on our RDIS dataset. Then we finetune the model on RGB-D datasets with metric ground-truth depths. During our finetuning, we formulate depth estimation as a classification task. This re-formulation scheme enables us to obtain the confidence of a depth prediction in the form of probability distribution. With this confidence, we propose an information gain loss to make use of the predictions that are close to ground-truth. We evaluate our approach on both indoor and outdoor benchmark RGB-D datasets and achieve state-of-the-art performance., 10 pages

We propose MCK-NET, a novel monocular framework for 3D target detection, location and pose estimation in autonomous driving scenarios, in which the key challenge is how to effectively mine depth information from a monocular image. To tackle this ill-posed problem, we combine the relative instance-depth of multiple corners in a monocular image to explicitly construct the corresponding depth relations between interest regions, from which MCK-NET learns to detect and locate objects based on geometric reasoning. In addition, there are two significant features existing in MCK-NET: One is to use the relative relationship between the 2D center and the 3D center projection to help locate the 3D center. The other is that the geometric constraints are established, including semantic keypoints, 2D box center and the projection of 3D center, which can enhance the 3D corner detection and improve the estimation accuracy of 3D centers. Experiments on the KITTI datasets show that MCK-NET achieves the most advanced results in all three tasks and outperforms the current monocular methods.

Visual evoked potentials (VEPs) assess the function of the visual pathway from the retina to the primary visual cortex. There is much evidence that monocular pattern-reversal and flash VEPs can distinguish dysfunction due to chiasmal and post-chiasmal afferent pathway lesions. There is less evidence about the use of pattern-onset/OFFset VEPs to identify post-chiasmic dysfunction. We present nine patients with a range of visual pathway defects that caused dense hemianopic field defects. These patients had pattern onset–OFFset VEPs recorded from an array of occipital electrodes referred to a mid-frontal electrode to checks that appeared for 230 ms and disappeared for 300 ms into a background of mean luminance, in a stimulus field of 30°. We found pattern-onset VEP components lateralise to occipital electrodes overlaying the functional hemisphere, whereas pattern-OFFset VEP components demonstrate the paradoxical lateralisation phenomenon, described in reversal VEPs, and are maximal over the contralateral occiput. Our findings show how extending the recording time window to include an OFFset VEP facilitates identification of hemianopic visual field defects. We advocate the pattern-onset/OFFset VEP in the assessment of patients with hemianopia, having particular value for patients who are otherwise unable to perform more demanding half-field electrophysiology, imaging or psychophysical testing.

Nowadays, robots equipped with a single camera, such as micro aerial vehicles (MAVs), are easily found at affordable costs. They can be used in different tasks, including the building of 3D environment maps. For building such maps, monocular simultaneous localization and mapping (SLAM) methods are employed, which usually generate sparse or semi-dense representations that are ill-suited for navigation tasks. We propose a new 3D exploration approach that uses a monocular camera as the only source of information. Our approach transforms a point cloud generated by monocular SLAM into local volumetric maps. These maps are built using the lines-of-sight between points and keyframes, allowing the MAV to navigate safely through the environment. Goal poses are dynamically defined to guide the MAV to explore the environment while avoiding obstacles. Besides that, the proposed approach seeks to determine properly when the environment was entirely explored, preventing that MAV stops before cover all the environment or flies more that is needed. The effectiveness of the proposed approach is evaluated in experiments in two different indoor environments, and show that it is possible to explore an environment using only a MAV equipped with a single monocular camera.

Advances in deep learning make monocular vision approaches attractive for the autonomous driving domain. This work investigates a method for estimating the speed of the ego-vehicle using state-of-the-art deep neural network based optical flow and single-view depth prediction models. Adopting a straightforward intuitive approach and approximating a single scale factor, several application schemes of the deep networks are evaluated and meaningful conclusions are formulated, such as: combining depth information with optical flow improves speed estimation accuracy as opposed to using optical flow alone; the quality of the deep neural network results influences speed estimation performance; using the depth and optical flow data from smaller crops of wide images degrades performance. With these observations in mind, a RMSE of less than 1 m/s for ego-speed estimation was achieved on the KITTI benchmark using monocular images as input. Limitations and possible future directions are discussed as well.

Predicting the depth of a scene from monocular infrared images, which plays a crucial role in understanding three-dimensional structures, is one of the challenging tasks in machine learning and computer vision. Considering the lack of texture and color information in infrared images, a novel discrete convolutional conditional random field network is proposed for depth estimation. The proposed method inherits several merits of conditional random fields and deep learning. First, the pairwise features are automatically extracted and optimized through deep architectures. Second, the monocular-images-based depth regression is converted into a multi-class classification, in which the order information of different levels of depths is considered in the loss function. Our experiments demonstrate that this conversion achieves much higher accuracy and faster conversion. Third, to obtain fine-grained level details, we have further proposed a multi-scale discrete convolutional conditional random field network that computes the pairwise features of the discrete conditional random field at different spatial levels. Extensive experiments on the infrared image dataset NUSTMS demonstrate that the proposed method outperforms other depth estimation methods. Specifically, for the proposed method, the mean relative error is 0.181, the mean log10 error is 0.072, and the accuracy with a threshold (t = 1.253) is 95.3%.

54 children with anisometropic amblyopia of ages 4-15 years were included in the study - 24 children in the patching and 30 children in the MFBF group. The study ran for a period of 6 months with BCVA assessed at each visit of follow up.In the patching group, visual acuity (VA) gain was 0.37 ± 0.26 log Mar over the course of 6 months with continuous improvement in vision across the three time points (0.20 log mar in the first month, 0.11 log Mar from month1 to 3 and 0.06 log Mar from months 3 to 6).In the Amb I Net group, visual acuity gain was 0.54 ± 0.38 log Mar over the same course of time. A continuous improvement in vision was also noted, with gains of 0.16 in the first month, 0.19 from month 1 to 3 and 0.19 from month 3 to 6. The difference in log Mar gains in VA was not statistically significant between the treatment arms and the final VA was similar (0.25 log Mar for patching and 0.18 log Mar for Amb I net). However, clinically the patching group showed 3 log MAR lines, whereas the MFBF group showed 5 log MAR lines of vision improvement at the end of therapy.MFBF was not superior to patching as far as efficacy is concerned in anisometropic amblyopia. However, the combination of more game options to sustain interest, a shorter duration of therapy over the course of a week and software directed compliance monitoring system favors the Amb I Net system from both patient and parental perspective. Amb I Net also negated discomfort, social stigma, and parental monitoring needed with patching regimens. It can be considered as a good alternative to patching in reluctant/non - compliant children with anisometropic amblyopia. This is the first large cohort study based on this therapy.

Attentional selection can be based on a particular location, feature, or object. In this study, we used a monocular cuing paradigm to investigate whether selective attention is based on the input's eye-of-origin. We found that unpredictive monocular flashing cues can trigger eye-based orienting when the cues are task-relevant. Specifically, the response to the target presented to the uncued location of the cued eye (same-eye condition) was faster than the response to the uncued location of the uncued eye (different-eye condition). This eye-specific attentional effect was mainly caused by the benefits in the same-eye condition rather than the costs in the different-eye condition. However, when the monocular cues were task-irrelevant, the eye-specific attentional effect vanished. Moreover, the monocular cues can modulate the spatial Stroop effect, leading to an increasing spatial Stroop effect at the attended eye, which is consistent with the typical modulation of endogenous attention on the spatial Stroop effect. Notably, participants were not aware of the eye-of-origin of flashing cues. These findings demonstrate that endogenous attentional selection can be based on the input's eye-of-origin without an individual's awareness of the input's eye-of-origin, suggesting that attentional systems can selectively cause the enhancement and exploration of visual information relevant to the current task in a specific monocular channel. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Heba M Shafik, Mohamed Ashraf Eldesouky, Dina Tadros Department of Ophthalmology, Faculty of Medicine, Tanta University, Tanta, EgyptCorrespondence: Heba M ShafikTanta University Hospital, El-Geish Street, Tanta, EgyptTel +00201227440829Fax +0020403274005Email drhebash81@gmail.comPurpose: To evaluate using the resected medial rectus muscle for elongation of the lateral rectus tendon in monocular surgery for large-angle sensory exotropia.Patients and Methods: A prospective case series was carried out in Tanta University hospital from January 2017 to June 2018 including 16 patients with large-angle sensory exotropia ≥ 50PD. The near and distance angles of exotropia were measured; any restrictions in adduction or abduction were recorded and scaled from − 4 to 0. Resection of the medial rectus muscle was done first; the resected segment was attached to the lateral rectus muscle after dissection by mattress sutures, and the muscle was recessed after elongation according to the preoperative distance angles. Successful motor alignment was considered if the angle measurement was within 10 PD of orthotropia at distance after 6 months. The stability of the alignment, any motility restrictions, and patient satisfaction were recorded at each follow-up visit until the 6th month.Results: The mean age of the study patients was 30.68± 13.30 years. The best-corrected visual acuity of the exotropic eyes by logMAR was 1.2± 0.3; the mean of the preoperative distance angle of deviation was − 72.19 ± 14.26 PD. The mean preoperative adduction and abduction restrictions were − 1.19 ± 1.52 and − 1.13 ± 1.09, respectively. There was significant improvement in the distance angle of deviation at the last follow-up after 6 months (p=0.001*). Also, there was improvement in both adduction and abduction restriction from the first to last follow-up. Thirteen patients (81.2%) were satisfied at the 6-month follow-up.Conclusion: Elongation of the lateral rectus muscle by an autograft from the resected medial rectus muscle is an easy, effective procedure for large-angle sensory exotropia. Patients achieve good postoperative alignment with minimal restriction of ocular motility.Keywords: sensory exotropia, autograft, lateral rectus elongation

Landmark/pose estimation in single monocular images have received much effort in computer vision due to its important applications. It remains a challenging task when input images severe occlusions caused by, e.g., adverse camera views. Under such circumstances, biologically implausible pose predictions may be produced. In contrast, human vision is able to predict poses by exploiting geometric constraints of landmark point inter-connectivity. To address the problem, by incorporating priors about the structure of pose components, we propose a novel structure-aware fully convolutional network to implicitly take such priors into account during training of the deep network. Explicit learning of such constraints is typically challenging. Instead, inspired by how human identifies implausible poses, we design discriminators to distinguish the real poses from the fake ones (such as biologically implausible ones). If the pose generator G generates results that the discriminator fails to distinguish from real ones, the network successfully learns the priors. Training of the network follows the strategy of conditional Generative Adversarial Networks (GANs). The effectiveness of the proposed network is evaluated on three pose-related tasks: 2D single human pose estimation, 2D facial landmark estimation and 3D single human pose estimation. The proposed approach significantly outperforms the state-of-the-art methods and almost always generates plausible pose predictions, demonstrating the usefulness of implicit learning of structures using GANs., 18 pages. Extended version of arXiv:1705.00389. Accepted to IEEE Trans. Pattern Analysis and Machine Intelligence

In recent years, the importance of measuring the three-dimensional position and orientation of manipulation targets in robot systems has been increasing. There is great demand to integrate robots with high-frame-rate vision systems to improve the working efficiency. A number of high-speed vision systems have been developed for high-frame-rate visual feedback. In this study, we focus on high-speed target tracking of both 3D position and orientation using only a monocular camera. This is a lightweight, low-cost solution for many fields, such as surgical navigation and drone avoidance. We modified the pixel-wise posterior 3D (PWP3D) framework and proposed a fast-PWP3D algorithm for high-speed target tracking and pose estimation. Unlike the original PWP3D method, our method improved the tracking speed (our current implementation runs at 400 Hz on a GPU board), and we showed that the tracking accuracy against changes in the environment (e.g., partial occlusion) was improved compared with the original PWP3D algorithm. By combining the fast-PWP3D algorithm with a visual servoing controller, we realized 500 Hz target tracking of both 3D position and orientation.

This article demonstrates the importance of visual information on depth perception from monocular motion parallax presented at the time of change in the motion direction of head and stimulus movements. In head-tracking systems, a longer delay time between the head and stimulus movements degrades the depth perception from the monocular motion parallax. Because this delay is noticeable at this time, we hypothesized that the visual information given at the time of the direction change plays a critical role in the depth perception from motion parallax. We evaluated depth perception from monocular motion parallax with and without a visual stimulus at the time of the motion direction change to confirm our hypothesis and clarified that stable and unambiguous depth can be perceived by presenting the change of the stimulus motion direction. We also demonstrated that it is the change in motion direction itself that is important rather than the temporal stop between deceleration and acceleration of the stimulus motion.

Each of our eyes sees a slightly different view of the physical world. Disparity is the small difference in position of features in the retinal images; stereopsis is the percept of depth from disparity. A distance between corresponding features in the retinal images of the two eyes smaller than the “upper disparity limit” yields a percept of depth; distances greater than this limit cause the two unfused monocular features to appear flattened into the fixation plane. This behavioral disparity limit is consistent with neurophysiological estimates of the largest disparity scale in primate, allowing us to relate physiological limits on plausible binocular interactions to separation between retinal locations. Here we test the hypothesis that this upper disparity limit predicts the presence of coarse stereopsis in humans with macular degeneration (MD), which affects the central retina but typically spares the periphery. The pattern of vision loss can be highly asymmetric, such that an intact location in one eye has a corresponding point in the other eye that lies within affected retina. Nevertheless, some individuals with MD have coarse stereopsis that is useful for eye-hand coordination. Our results show that individuals with MD (n = 25, male and female) have coarse stereopsis when the distance between intact retinal locations is less than the behavioral and physiological upper disparity limit at the corresponding eccentricity. Furthermore, for those without stereopsis, we can predict whether they can achieve stereopsis by using alternate retinal loci at further eccentricities whose separation is below the upper disparity limit. SIGNIFICANCE STATEMENT We show that the largest separation between features in the two eyes that yields a percept of depth in humans is related to the largest disparity scale in macaque medial temporal area and to the estimated size of the receptive fields in human depth-sensitive cortical regions. This upper disparity limit also predicts whether individuals with retinal damage due to macular degeneration will have stereopsis. Individuals have stereopsis when the separation between intact retinal locations in the two eyes is smaller than the upper disparity limit measured behaviorally. Our results indicate the importance of the behavioral upper disparity limit as a predictor for stereopsis in populations with retinal damage.

Analysis of state-of-the-art VO/VSLAM system exposes a gap in balancing performance (accuracy & robustness) and efficiency (latency). Feature-based systems exhibit good performance, yet have higher latency due to explicit data association; direct & semidirect systems have lower latency, but are inapplicable in some target scenarios or exhibit lower accuracy than feature-based ones. This paper aims to fill the performance-efficiency gap with an enhancement applied to feature-based VSLAM. We present good feature matching, an active map-to-frame feature matching method. Feature matching effort is tied to submatrix selection, which has combinatorial time complexity and requires choosing a scoring metric. Via simulation, the Max-logDet matrix revealing metric is shown to perform best. For real-time applicability, the combination of deterministic selection and randomized acceleration is studied. The proposed algorithm is integrated into monocular & stereo feature-based VSLAM systems. Extensive evaluations on multiple benchmarks and compute hardware quantify the latency reduction and the accuracy & robustness preservation., Accepted as a Regular Paper to the IEEE Transactions on Robotics Journal

Monocular visual-inertial measurement unit (IMU) odometry has been widely used in various intelligent vehicles. As a popular technique, detector–descriptor-based visual-IMU odometry is effective and efficient due to the fact that local descriptors are robust against occlusions, background clutter, and abrupt content changes. However, to our knowledge, there is not a comprehensive and comparative evaluation study on the performance of different combinations of detectors and descriptors recently developed. In order to bridge this gap, we conduct such a comparative study in a unified framework. In particular, six typical routes with different lengths, shapes, and road scenes are selected from the well-known KITTI dataset. We first evaluate the performance of different combinations of salient point detectors and local descriptors using the six routes. Then, we tune the parameters of the best detector or descriptor obtained for each route, to further augment the results. This paper provides not only comprehensive benchmarks for assessing various algorithms but also instructive guidelines and insights for developing detectors and descriptors to handle different road scenes.

AIM: To observe the changes of binocular eye accommodation function in correcting anisometropia by monocular orthokeratology in anisometropic children, and to explore the mechanism of regulation in the progression of myopia control by orthokeratology.METHODS: It was a prospective self-control study. A total of 22 anisometropic children(44 eyes), aged 8-13 years, who wore an orthokeratology lens in only one eye were enrolled in the outpatient clinic of Hainan Provincial Eye Hospital from September 2016 to September 2018. The eyes with an orthokeratology lens had a mean spherical equivalent refraction of -2.751±1.16 diopter(D), and fellow eyes without a lens had a refraction of -0.10±0.32D. The eyes wearing orthokeratology lens were used as the lens wearing group, and the fellow eyes without a lens were used as non-wearing lens group. During the follow-up period,we measured changes of diopter difference, monocular accommodative amplitude(MAA), monocular accommodative facility(MAF)and accommodative response(AR)before wearing the lens and wearing the lens for 12mo.RESULTS: The monocular accommodative amplitude and monocular accommodative facility of the wearing group before wearing the lens were lower than those of the non-wear group(PPt=3.63, Pt=0.75, P=0.46)compared with the non-wearing group. The monocular accommodative facility 14.63±1.58 cyc/min after wearing the lens for 12mo was higher than that before wearing the lens 9.25±3.38cyc/min(t=2.83, P=0.01), and the difference was not statistically significant(t=0.38, P=0.71)compared with the non-wearing group. The accommodative lag 0.62±0.29D of the wearing group after 12mo was reduced compared with that before wearing the lens 1.35±0.26D(t=2.57, P=0.02),and compared with the non-wearing group, the difference was not statistical significance(t=0.61, P=0.55). The mean equivalent diopter of the non-lens group was -0.75±0.35D after 12mo, and the number of diopter group increased by -0.65±0.39D(t=4.24, Pt=2.90, Pt=5.30, PP>0.05).CONCLUSION: The accommodation function of anisometropic children in correcting anisometropia by monocular orthokeratology was significantly improved compared with that before wearing the lens and was consistent with the contralateral eyes without lens group. The refractive state of the group wearing orthokeratology lens was more stable, and the contralateral eyes without lens gradually showed mild myopia refractive state, but the accommodation did not change significantly.

BACKGROUND High-contrast visual acuity (HCVA) changes with age, yet little is known about pediatric-specific age- and sex-normative values for low-contrast letter acuity (LCLA). We define maturational changes in monocular and binocular HCVA and LCLA in childhood and adolescence. METHODS Normally sighted youth (ages 5-20 years, without neurologic or ophthalmologic disease and best-corrected HCVA of 20/25 or better in each eye) were recruited. Mean monocular and binocular scores using Early Treatment Diabetic Retinopathy Study (for HCVA) and 2.5% and 1.25% Sloan (for LCLA) charts and the magnitude of binocular summation were calculated using 2-year bins. Relationships between scores and age were explored using scatterplots with Locally Weighted Scatterplot Smoothing (LOWESS) and analysis of variance that accounts for intereye correlation, followed by test of linear trend for age effect. RESULTS Among 101 (202 eyes) healthy participants (mean age 13 years, 42% males), monocular and binocular scores varied by age, with highest mean scores achieved in the 13 to 14-year age group for both HCVA and LCLA. Between the ages of 5 and 14.9 years, monocular scores increased linearly with age (0.76 letter/year for HCVA, 1.11 letters/year for 2.5% LCLA, and 0.97 letter/year for 1.25% LCLA; all P < 0.0001). Binocular HCVA scores also increased with age between 5 and 14.9 years (0.71 letters/year, P < 0.0001). The magnitude of binocular summation for HCVA or LCLA did not change with age. CONCLUSIONS HCVA and LCLA abilities mature into adolescence, peak between 13 and 14.9 years of age, and then plateau into adulthood. Evaluation of patients with visual deficits should consider age-expected normal visual acuity.

Monocular depth estimation by unsupervised learning is a potential strategy, which is mainly self-supervised by calculating view reconstruction loss from stereo pairs or monocular sequences. However, most existing works only consider the geometric information during training, without using semantics. We propose a semantic monocular depth estimation (SE-Net), a neural network framework that estimates depth using semantic information and video sequences. The whole framework is semi-supervised, because we take advantage of labelled semantic ground truth data. In view of the structural consistency between the semantically segmented image and the depth map, we first perform semantic segmentation on the image, and then use the semantic labels to guide the construction of the depth estimation network. Experiments on the KITTI dataset show that learning semantic information from images can effectively improve the effect of monocular depth estimation, and SE-Net is superior to the most advanced methods in depth estimation accuracy.

The purpose of this study was to investigate the benefits of vision training with visual evoked potentials (VEP) biofeedback in amblyopia after the critical period in 8 to 17-year (11.5 ± 3.1) old children. Ten participants with monocular amblyopia after the critical period underwent a 10-week, 20-session vision training program with the Retimax Vision Trainer device. During each session, the participants were instructed to be as focused as possible onto the fixation point in the middle of the screen. The size of the fixation point and the pitch of the background sound were changing according to VEP parameters and thus provided the participants real-time feedback of their visual performance. The mean BCVA improvement across our group was 0.12 LogMAR (p

SUMMARYWe present a 3D reconstruction method using brightness and camera motion estimation for registering local colon structure in colonoscopy. The proposed method is based on reverse projection from 2D fold contours to 3D space, motion estimation from 3D reconstructed points between neighboring frames, and model registration to reconstruct the fold structure. On the synthetic colon, the average percentages of the reconstructed depth error and circumference error are about 14.2% and 15.2%, respectively. The accuracy is enough for the navigation and control in capsule robot. This work demonstrates that the proposed method is superior to the methods using single-frame-based brightness intensity.

Vehicle detection and distance estimation are critical components of driver assistance system and self-driving system, and considerable different frameworks have been investigated such as radar, laser and camera-based. Among them, camera-based vehicle detection and distance estimation have an obvious advantage over other systems in that it needs lower cost. However, existing camera-based methods are not robust enough under complex driving scenes. In this work, an end-to-end deep convolutional neural network framework is proposed to jointly detect vehicles and estimate vehicle distance efficiently. Specifically, a monocular depth estimation method is designed to transform the RGB appearance information into depth modality information. Then the vehicle detection module takes the RGB and depth image as inputs to improve the detection performance. Finally, the distance estimation module employs the detection results and the estimated depth information to predict the distance more precisely. The whole network can be trained in an end-to-end manner with the multi-task loss function. The proposed framework is evaluated on the public vehicle detection benchmark KITTI to show the effectiveness of the proposed framework. Moreover, the performance of three proposed sub-modules are also analysed separately to give a more comprehensive evaluation of the designed framework.

When similar visual stimuli are presented binocularly to both eyes, one perceives a fused single image. However, when the two stimuli are distinct, one does not perceive a single image; instead, one perceives binocular rivalry. That is, one perceives one of the stimulated patterns for a few seconds, then the other for few seconds, and so on - with random transitions between the two percepts. Most theoretical studies focus on rivalry, with few considering the coexistence of fusion and rivalry. Here we develop three distinct computational neuronal network models which capture binocular rivalry with realistic stochastic properties, fusion, and the hysteretic transition between. Each is a conductance-based point neuron model, which is multi-layer with two ocular dominance columns (L & R) and with an idealized "ring" architecture where the orientation preference of each neuron labels its location on a ring. In each model, the primary mechanism initiating binocular rivalry is cross-column inhibition, with firing rate adaptation governing the temporal properties of the transitions between percepts. Under stimulation by similar visual patterns, each of three models uses its own mechanism to overcome cross-column inhibition, and thus to prevent rivalry and allow the fusion of similar images: The first model uses cross-column feedforward inhibition from the opposite eye to "shut off" the cross-column feedback inhibition; the second model "turns on" a second layer of monocular neurons as a parallel pathway to the binocular neurons, rivaling out of phase with the first layer, and together these two pathways represent fusion; and the third model uses cross-column excitation to overcome the cross-column inhibition and enable fusion. Thus, each of the idealized ring models depends upon a different mechanism for fusion that might emerge as an underlying mechanism present in real visual cortex.

We present a self-supervised approach to training convolutional neural networks for dense depth estimation from monocular endoscopy data without a priori modeling of anatomy or shading. Our method only requires monocular endoscopic videos and a multi-view stereo method, e.g., structure from motion, to supervise learning in a sparse manner. Consequently, our method requires neither manual labeling nor patient computed tomography (CT) scan in the training and application phases. In a cross-patient experiment using CT scans as groundtruth, the proposed method achieved submillimeter mean residual error. In a comparison study to recent self-supervised depth estimation methods designed for natural video on in vivo sinus endoscopy data, we demonstrate that the proposed approach outperforms the previous methods by a large margin. The source code for this work is publicly available online at https://github.com/lppllppl920/EndoscopyDepthEstimation-Pytorch., Comment: Accepted to IEEE Transactions on Medical Imaging

AIM: To study the curative efficacy of different surgical procedures in the treatment of congenital unilateral monocular oblique paralysis and its effects on binocular vision function.METHODS: Totally 82 children of congenital unilateral monocular oblique paralysis who received therapy from May 2016 to 2019 in our hospital were selected as research objects, operative plan was determined by the degree of hyperfunction of the inferior oblique muscle, the movement of the eyeball and the vertical obliquity of the eye position, the inferior oblique muscle tendon amputation(3 cases), partial inferior oblique muscle resection(63 cases), inferior oblique muscle recession(6 cases), posterior rectus muscle repair(4 cases), inferior oblique muscle weakening+contralateral/ipsilateral rectus muscle operation(5 cases), upper oblique folding(1 cases). The effects of different surgical procedures and the changes of binocular visual function, corrected visual acuity and compensatory head position before and after surgery were compared. RESULTS: Compared with before surgery, the fusion function, hyperstereopsis, myopia, corrected visual acuity and compensatory head position were significantly improved after operation(PP>0.05). CONCLUSION: Choosing different surgical methods according to the severity of congenital superior oblique paralysis has positive significance, which can improve the vision and compensating head position of children, and it's helpful to rebuild binocular vision function.

The human eye is capable of responding to the presence of blur by changing its axial length, so that the retina moves towards the defocused image plane. We measured how quickly the eye length changed in response to both myopic and hyperopic defocus and how quickly the eye length changed when the defocus was removed. Axial length was measured at baseline and every 10 minutes during 1 hour of exposure to monocular defocus (right eye) with the left eye optimally corrected for two defocus conditions (+3 D and −3 D) and a control condition. Recovery was measured for 20 minutes after blur removal. A rapid increase in axial length was observed after exposure (~2 minutes) to hyperopic defocus (+7 ± 5 μm, p

This paper proposes a 3D reconstruction scheme for monocular cameras based on an improved line structure cursor positioning method and the Scheimpflug principle to overcome the limitations of current calibration methods for line-structured light 3D reconstruction and binocular stereo vision matching. Unlike the traditional line structure cursor positioning method, the line-structured light is projected on a blank area of a high-precision chessboard target to avoid the low corner point extraction accuracy caused by the intersection of the structured light strip and the chessboard target. The 3D coordinates of the point in the camera coordinate system can be obtained by combining the linear equation between the origin and the center point of the light stripe on the target and the plane equation of the target. The light plane equation is obtained through data fitting on the basis of the 3D information of the light stripe center of two or more pictures. The experimental system allows the position relationship of the light source, lens, and complementary metal oxide semiconductor to meet the Scheimpflug conjugate’s clear imaging conditions for improving the visual range of the measurement system. Results show that the measurement system obtains good measurement accuracy and robustness.

Different species show an intriguing similarity in representing numerosity in space, starting from left to right. This bias has been attributed to a right hemisphere dominance in processing spatial information. Here, to disentangle the role of each hemisphere in dealing with spatial versus ordinal-numerical information, we tested domestic chicks during monocular versus binocular vision. In the avian brain, the contralateral hemisphere mainly processes the visual input from each eye. Four-day-old chicks learned to peck at the fourth element in a sagittal series of 10 identical elements. At testing, chicks faced a left-to-right-oriented series where the interelement distance was manipulated so that the third element was where the fourth had been at training; this compelled chicks to use either spatial or ordinal cues. Chicks tested binocularly selected both the fourth left and (to a lesser extent) right elements. Chicks tested monocularly chose the third and fourth elements on the seeing side equally. Interhemispheric cooperation resulted in the use of ordinal-numerical information, while each single hemisphere could rely on spatial or ordinal-numerical cue. Both hemispheres can process spatial and ordinal-numerical information, but their interaction results in the supremacy of processing the ordinal-numerical cue.

The neural network based approach for 3D human pose estimation from monocular images has attracted growing interest. However, annotating 3D poses is a labor-intensive and expensive process. In this paper, we propose a novel self-supervised approach to avoid the need of manual annotations. Different from existing weakly/self-supervised methods that require extra unpaired 3D ground-truth data to alleviate the depth ambiguity problem, our method trains the network only relying on geometric knowledge without any additional 3D pose annotations. The proposed method follows the two-stage pipeline: 2D pose estimation and 2D-to-3D pose lifting. We design the transform re-projection loss that is an effective way to explore multi-view consistency for training the 2D-to-3D lifting network. Besides, we adopt the confidences of 2D joints to integrate losses from different views to alleviate the influence of noises caused by the self-occlusion problem. Finally, we design a two-branch training architecture, which helps to preserve the scale information of re-projected 2D poses during training, resulting in accurate 3D pose predictions. We demonstrate the effectiveness of our method on two popular 3D human pose datasets, Human3.6M and MPI-INF-3DHP. The results show that our method significantly outperforms recent weakly/self-supervised approaches.

Monocular 3D object detection task aims to predict the 3D bounding boxes of objects based on monocular RGB images. Since the location recovery in 3D space is quite difficult on account of absence of depth information, this paper proposes a novel unified framework which decomposes the detection problem into a structured polygon prediction task and a depth recovery task. Different from the widely studied 2D bounding boxes, the proposed novel structured polygon in the 2D image consists of several projected surfaces of the target object. Compared to the widely-used 3D bounding box proposals, it is shown to be a better representation for 3D detection. In order to inversely project the predicted 2D structured polygon to a cuboid in the 3D physical world, the following depth recovery task uses the object height prior to complete the inverse projection transformation with the given camera projection matrix. Moreover, a fine-grained 3D box refinement scheme is proposed to further rectify the 3D detection results. Experiments are conducted on the challenging KITTI benchmark, in which our method achieves state-of-the-art detection accuracy., Comment: 11 pages, 8 figures, AAAI2020