Search

Addressing accuracy limitations and pose ambiguity in 6D object pose estimation from single RGB images presents a significant challenge, particularly due to object symmetries or occlusions. In response, we introduce a novel score-based diffusion method applied to the $SE(3)$ group, marking the first application of diffusion models to $SE(3)$ within the image domain, specifically tailored for pose estimation tasks. Extensive evaluations demonstrate the method's efficacy in handling pose ambiguity, mitigating perspective-induced ambiguity, and showcasing the robustness of our surrogate Stein score formulation on $SE(3)$. This formulation not only improves the convergence of Langevin dynamics but also enhances computational efficiency. Thus, we pioneer a promising strategy for 6D object pose estimation., Comment: Preprint

Implicit neural representation has recently shown a promising ability in representing images with arbitrary resolutions. In this paper, we present a Local Implicit Transformer (LIT), which integrates the attention mechanism and frequency encoding technique into a local implicit image function. We design a cross-scale local attention block to effectively aggregate local features. To further improve representative power, we propose a Cascaded LIT (CLIT) that exploits multi-scale features, along with a cumulative training strategy that gradually increases the upsampling scales during training. We have conducted extensive experiments to validate the effectiveness of these components and analyze various training strategies. The qualitative and quantitative results demonstrate that LIT and CLIT achieve favorable results and outperform the prior works in arbitrary super-resolution tasks.

This paper introduces pixel-wise prediction based visual odometry (PWVO), which is a dense prediction task that evaluates the values of translation and rotation for every pixel in its input observations. PWVO employs uncertainty estimation to identify the noisy regions in the input observations, and adopts a selection mechanism to integrate pixel-wise predictions based on the estimated uncertainty maps to derive the final translation and rotation. In order to train PWVO in a comprehensive fashion, we further develop a data generation workflow for generating synthetic training data. The experimental results show that PWVO is able to deliver favorable results. In addition, our analyses validate the effectiveness of the designs adopted in PWVO, and demonstrate that the uncertainty maps estimated by PWVO is capable of capturing the noises in its input observations.

Photovoltaic cells for indoor light harvesting have recently attracted considerable attention for low power consumption electronic products due to the rapid growth of the Internet of Things. Dye-sensitized cells (DSC) show tremendous promise in delivering a low-cost, high-manufacturability, and excellent conversion efficiency for various artificial lights to combine with electronics achieving the energy autonomous system. Commercial DSC technology in terms of the reproducibility on cell efficiency and lifetime prediction was discussed to meet requirements of photovoltaic cell market for various applications. The DSC automated pilot line towards industrially relevant manufacturing volumes was described, providing an average efficiency of 13.8% at 1000 lux white LED light and an overall yield of 98.7%. Furthermore, an approach to obtain the database of degradation rate constant for various environmental stresses was proposed according to Arrhenius plots and can be used to predict DSC lifetime for a variety of situations. The lifetime was evaluated to 10.9 years for the indoor situation using the corresponding degradation rate constant.

Ethylene perceived by a family of five receptors regulates many developmental processes in Arabidopsis. Here we conducted the yeast two-hybrid assay to screen for additional unidentified proteins that interact with subfamily II ethylene receptor ETR2. Three SAUR proteins, named SAUR76, 77 and 78, were identified to associate with both ETR2 and EIN4 in different assays. Interaction of SAUR76 and SAUR78 with ETR2 was further verified by co-immunoprecipitation and bimolecular fluorescence complementation (BiFC) assays. Expressions of SAUR76-78 are induced by auxin and ethylene treatments. Compared with wild type, SAUR-overexpressing plants exhibit reduced ethylene sensitivity, while SAUR-RNAi lines exhibit enhanced ethylene sensitivity. Overexpressing the three SAURs partially complements the phenotype of subfamily II ethylene receptor loss-of-function double mutant etr2-3ein4-4, which has increased ethylene response and small cotyledon and rosette. saur76 mutation partially suppresses the reduced ethylene sensitivity of etr2-2. SAUR76/78 proteins are regulated by 26S proteasome system and larger tag increases their protein stability. These findings suggest that SAUR76-78 may affect ethylene receptor signaling and promote plant growth in Arabidopsis.

Open procedures represent the dominant form of surgery worldwide. Artificial intelligence (AI) has the potential to optimize surgical practice and improve patient outcomes, but efforts have focused primarily on minimally invasive techniques. Our work overcomes existing data limitations for training AI models by curating, from YouTube, the largest dataset of open surgical videos to date: 1997 videos from 23 surgical procedures uploaded from 50 countries. Using this dataset, we developed a multi-task AI model capable of real-time understanding of surgical behaviors, hands, and tools - the building blocks of procedural flow and surgeon skill. We show that our model generalizes across diverse surgery types and environments. Illustrating this generalizability, we directly applied our YouTube-trained model to analyze open surgeries prospectively collected at an academic medical center and identified kinematic descriptors of surgical skill related to efficiency of hand motion. Our Annotated Videos of Open Surgery (AVOS) dataset and trained model will be made available for further development of surgical AI., Comment: 22 pages, 4 main text figures, 7 extended data figures, 4 extended data tables

We propose a novel depth maps refinement algorithm and generate multi-view video sequences from two-view video sequences for modern autostereoscopic display. In order to generate realistic contents for virtual views, high-quality depth maps are very critical to the view synthesis results. Therefore, refining the depth maps is the main challenging problem in the task. We propose an iterative depth refinement algorithm, including error detection and error correction, to correct errors in depth map. Error detection aims at two types of error: across-view color-depth-inconsistency error and local color-depth-inconsistency error. Then, error pixels are corrected based on sampling local candidates. A trilateral filter that considers intensity, spatial and temporal terms into the filter weighting is applied to enhance the spatial and temporal consistency across frames. So the virtual views can be better synthesized according to the refined depth maps. To combine both warped images, disparity-based view interpolation is introduced to alleviate the translucent artifacts. Finally, a directional filter is applied to reduce the aliasing around the object boundaries to generate multiple high-quality virtual views between the two views. We demonstrate the superior image quality of the synthesized virtual views by using the proposed algorithm over the state-of-the-art view synthesis methods through experiments on benchmarking image and video datasets.

For the resonant grounding system, the existing single criterion is less applicable, and the data fusion method based on intelligent algorithm will be restricted by the training sample, therefore, a intelligent fault line selection method based on S transform direction and energy is proposed in this paper. Firstly, with the powerful signal extraction ability of the S transform algorithm, the characteristic band signals are extracted from the zero sequence currents of all lines. Then the phase difference and energy parameters of each line are calculated. Finally, the parameters of the energy parameters are processed with the difference of the angle difference as the weight number, which achieves the purpose of merging two criteria, and enlarges the fault boundary, and the line with the most comprehensive parameters is the fault line. The simulation results show that the method has high margin and wide application scope, and is less affected by fault resistance, fault closing angle and fault location.

A cruciform cavity is presented for multi-wavelength laser generation. On the basis of considering the optimal power ratio and good spatial overlap of the two fundamental beams, the maximum output power of 589 nm laser reaches 3.5 W when the pumping power of Nd:YAG A and Nd:YAG B are 311.5 W and 261.8 W, respectively. At the same time, the other wavelength lasers are also obtained with the output power distribution of 2.5 W at 660nm, 15W at 532nm, 100m W at 1319nm and 240mW at 1064nm. The corresponding beam quality factors are Mx2 = 4.93, My2 = 5.01 at 589 nm, Mx2 = 4.51, My2 = 4.85 at 660 nm, and Mx2 = 4.12, My2 = 3.96 at 532nm, respectively. The instabilities of the three visible lights are measured, which are also less than 2% within three hours.

Ethylene is an important phytohormone in the regulation of plant growth, development, and stress response throughout the lifecycle. Previously, we discovered that a subfamily II ethylene receptor tobacco (Nicotiana tabacum) Histidine Kinase1 (NTHK1) promotes seedling growth. Here, we identified an NTHK1-interacting protein translationally controlled tumor protein (NtTCTP) by the yeast (Saccharomyces cerevisiae) two-hybrid assay and further characterized its roles in plant growth. The interaction was further confirmed by in vitro glutathione S-transferase pull down and in vivo coimmunoprecipitation and bimolecular fluorescence complementation assays, and the kinase domain of NTHK1 mediates the interaction with NtTCTP. The NtTCTP protein is induced by ethylene treatment and colocalizes with NTHK1 at the endoplasmic reticulum. Overexpression of NtTCTP or NTHK1 reduces plant response to ethylene and promotes seedling growth, mainly through acceleration of cell proliferation. Genetic analysis suggests that NtTCTP is required for the function of NTHK1. Furthermore, association of NtTCTP prevents NTHK1 from proteasome-mediated protein degradation. Our data suggest that plant growth inhibition triggered by ethylene is regulated by a unique feedback mechanism, in which ethylene-induced NtTCTP associates with and stabilizes ethylene receptor NTHK1 to reduce plant response to ethylene and promote plant growth through acceleration of cell proliferation.

We propose a novel depth maps refinement algorithm and generate multi-view video sequences from two-view video sequences for modern autostereoscopic display. In order to generate realistic contents for virtual views, high-quality depth maps are very critical to the view synthesis results. We propose an iterative depth refinement approach of a joint error detection and correction algorithm to refine the depth maps that can be estimated by an existing stereo matching method or provided by a depth capturing device. Error detection aims at two types of error: across-view color-depth-inconsistency error and local color-depth-inconsistency error. Subsequently, the detected error pixels are corrected by searching appropriate candidates under several constraints to amend the depth errors. A trilateral filter is included in the refining process that considers intensity, spatial and temporal terms into the filter weighting to enhance the consistency across frames. In the proposed view synthesis framework, it features a disparity-based view interpolation method to alleviate the translucent artifacts and a directional filter to reduce the aliasing around the object boundaries. Experimental results show that the proposed algorithm effectively fixes errors in the depth maps. In addition, we also show the refined depth maps along with the proposed view synthesis framework significantly improve the novel view synthesis on several benchmark dataseis.

Operation in minimally invasive surgery is more difficult since the surgeons perform operations without haptic feedback or depth perception. Moreover, the field of view perceived by the surgeons through endoscopy is usually quite limited. The goal of this paper is to allow surgeons to see wide-angle images from endoscopy without the drawback of lens distortion. The proposed distortion correction process consists of lens calibration and real-time image warping. The calibration step is to estimate the parameters in the lens distortion model. We propose a fully automatic Hough-entropy-based calibration algorithm, which provides calibration results comparable to the previous manual calibration method. To achieve real-time correction, we use graphics processing unit to warp the image in parallel. In addition, surgeons may adjust the focal length of a lens during the operation. Real-time distortion correction of a zoomable lens is impossible by using traditional calibration methods because the tedious calibration process has to repeat again if focal length is changed. We derive a formula to describe the relationship between the distortion parameter, focal length, and image boundary. Hence, we can estimate the focal length for a zoomable lens from endoscopic images online and achieve real-time lens distortion correction.

In this paper we present a novel view synthesis method for mobile platform. A disparity-based view interpolation is proposed to synthesize the virtual view from a left and right image pair. This makes to full use of the available information of both images to give accurate interpolation results and greatly decreases the pixel number in the disoccusion region, thus reducing the errors introduced by the patch-based image inpainting. Two boundary refining schemes considering gradient and color coherence and applying directional filters on boundaries are proposed to improve the synthesized results. Experimental results show that the proposed method is effective and suitable for mobile environment.

Touch screen as a kind of simple and convenient input device has been already used in consumer electronics and industrial control areas. S3C2410 is a embedded processor, which is with the ARM920T core produced by Samsung Corp. This paper introduces the architecture of touch screen and its driver under Linux based on ADS7843 connected with S3C2410 platform. Both hardware architecture and software design of the system are described.

Wide-angle lens provides a broad field of view which benefits some applications, such as video surveillance or endoscopic imaging. However, it also induces lens distortion, especially radial distortion, which may impede further video analysis or perceptual interpretation. For some applications, such as minimally invasive surgery and visual surveillance, real-time correction of image distortion is required. Traditional CPU-centric machines are difficult to achieve the requirement of real-time computation because of a large amount of computation. In this paper, we propose to achieve real-time correction of wide-angle lens distortion of images on several target platforms. In the GPGPU platform, we achieve real-time correction at full-HD resolution by using CUDA. For middle-end devices, an error-controllable mesh is used and the system is implemented by industry standard, OpenGL. We also implement it with OpenGL ES on embedded GPUs for mobile devices. Experiments show using our error-controllable mesh greatly outperforms those using regularly downsampled meshes.