Search

This paper reports on the NTIRE 2023 Quality Assessment of Video Enhancement Challenge, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2023. This challenge is to address a major challenge in the field of video processing, namely, video quality assessment (VQA) for enhanced videos. The challenge uses the VQA Dataset for Perceptual Video Enhancement (VDPVE), which has a total of 1211 enhanced videos, including 600 videos with color, brightness, and contrast enhancements, 310 videos with deblurring, and 301 deshaked videos. The challenge has a total of 167 registered participants. 61 participating teams submitted their prediction results during the development phase, with a total of 3168 submissions. A total of 176 submissions were submitted by 37 participating teams during the final testing phase. Finally, 19 participating teams submitted their models and fact sheets, and detailed the methods they used. Some methods have achieved better results than baseline methods, and the winning methods have demonstrated superior prediction performance.

Modern consumer cameras usually employ the rolling shutter (RS) mechanism, where images are captured by scanning scenes row-by-row, yielding RS distortions for dynamic scenes. To correct RS distortions, existing methods adopt a fully supervised learning manner, where high framerate global shutter (GS) images should be collected as ground-truth supervision. In this paper, we propose a Self-supervised learning framework for Dual reversed RS distortions Correction (SelfDRSC), where a DRSC network can be learned to generate a high framerate GS video only based on dual RS images with reversed distortions. In particular, a bidirectional distortion warping module is proposed for reconstructing dual reversed RS images, and then a self-supervised loss can be deployed to train DRSC network by enhancing the cycle consistency between input and reconstructed dual reversed RS images. Besides start and end RS scanning time, GS images at arbitrary intermediate scanning time can also be supervised in SelfDRSC, thus enabling the learned DRSC network to generate a high framerate GS video. Moreover, a simple yet effective self-distillation strategy is introduced in self-supervised loss for mitigating boundary artifacts in generated GS images. On synthetic dataset, SelfDRSC achieves better or comparable quantitative metrics in comparison to state-of-the-art methods trained in the full supervision manner. On real-world RS cases, our SelfDRSC can produce high framerate GS videos with finer correction textures and better temporary consistency. The source code and trained models are made publicly available at https://github.com/shangwei5/SelfDRSC., Comment: 16 pages, 16 figures, available at https://github.com/shangwei5/SelfDRSC

Various depth estimation models are now widely used on many mobile and IoT devices for image segmentation, bokeh effect rendering, object tracking and many other mobile tasks. Thus, it is very crucial to have efficient and accurate depth estimation models that can run fast on low-power mobile chipsets. In this Mobile AI challenge, the target was to develop deep learning-based single image depth estimation solutions that can show a real-time performance on IoT platforms and smartphones. For this, the participants used a large-scale RGB-to-depth dataset that was collected with the ZED stereo camera capable to generated depth maps for objects located at up to 50 meters. The runtime of all models was evaluated on the Raspberry Pi 4 platform, where the developed solutions were able to generate VGA resolution depth maps at up to 27 FPS while achieving high fidelity results. All models developed in the challenge are also compatible with any Android or Linux-based mobile devices, their detailed description is provided in this paper., arXiv admin note: substantial text overlap with arXiv:2105.08630, arXiv:2211.03885; text overlap with arXiv:2105.08819, arXiv:2105.08826, arXiv:2105.08629, arXiv:2105.07809, arXiv:2105.07825

The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.

The role of mobile cameras increased dramatically over the past few years, leading to more and more research in automatic image quality enhancement and RAW photo processing. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based image signal processing (ISP) pipeline replacing the standard mobile ISPs that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale Fujifilm UltraISP dataset consisting of thousands of paired photos captured with a normal mobile camera sensor and a professional 102MP medium-format FujiFilm GFX100 camera. The runtime of the resulting models was evaluated on the Snapdragon's 8 Gen 1 GPU that provides excellent acceleration results for the majority of common deep learning ops. The proposed solutions are compatible with all recent mobile GPUs, being able to process Full HD photos in less than 20-50 milliseconds while achieving high fidelity results. A detailed description of all models developed in this challenge is provided in this paper.

As mobile cameras with compact optics are unable to produce a strong bokeh effect, lots of interest is now devoted to deep learning-based solutions for this task. In this Mobile AI challenge, the target was to develop an efficient end-to-end AI-based bokeh effect rendering approach that can run on modern smartphone GPUs using TensorFlow Lite. The participants were provided with a large-scale EBB! bokeh dataset consisting of 5K shallow / wide depth-of-field image pairs captured using the Canon 7D DSLR camera. The runtime of the resulting models was evaluated on the Kirin 9000's Mali GPU that provides excellent acceleration results for the majority of common deep learning ops. A detailed description of all models developed in this challenge is provided in this paper., Comment: arXiv admin note: substantial text overlap with arXiv:2211.03885; text overlap with arXiv:2105.07809, arXiv:2211.04470, arXiv:2211.05256, arXiv:2211.05910

Cameras capture sensor RAW images and transform them into pleasant RGB images, suitable for the human eyes, using their integrated Image Signal Processor (ISP). Numerous low-level vision tasks operate in the RAW domain (e.g. image denoising, white balance) due to its linear relationship with the scene irradiance, wide-range of information at 12bits, and sensor designs. Despite this, RAW image datasets are scarce and more expensive to collect than the already large and public RGB datasets. This paper introduces the AIM 2022 Challenge on Reversed Image Signal Processing and RAW Reconstruction. We aim to recover raw sensor images from the corresponding RGBs without metadata and, by doing this, "reverse" the ISP transformation. The proposed methods and benchmark establish the state-of-the-art for this low-level vision inverse problem, and generating realistic raw sensor readings can potentially benefit other tasks such as denoising and super-resolution., Comment: ECCV 2022 Advances in Image Manipulation (AIM) workshop

This paper introduces the methods and the results of AIM 2022 challenge on Instagram Filter Removal. Social media filters transform the images by consecutive non-linear operations, and the feature maps of the original content may be interpolated into a different domain. This reduces the overall performance of the recent deep learning strategies. The main goal of this challenge is to produce realistic and visually plausible images where the impact of the filters applied is mitigated while preserving the content. The proposed solutions are ranked in terms of the PSNR value with respect to the original images. There are two prior studies on this task as the baseline, and a total of 9 teams have competed in the final phase of the challenge. The comparison of qualitative results of the proposed solutions and the benchmark for the challenge are presented in this report., Comment: 14 pages, 9 figures, Challenge report of AIM 2022 Instagram Filter Removal Challenge in conjunction with ECCV 2022

Semantic segmentation is an important task in computer vision, from which some important usage scenarios are derived, such as autonomous driving, scene parsing, etc. Due to the emphasis on the task of video semantic segmentation, we participated in this competition. In this report, we briefly introduce the solutions of team 'BetterThing' for the ICCV2021 - Video Scene Parsing in the Wild Challenge. Transformer is used as the backbone for extracting video frame features, and the final result is the aggregation of the output of two Transformer models, SWIN and VOLO. This solution achieves 57.3% mIoU, which is ranked 3rd place in the Video Scene Parsing in the Wild Challenge.

Microporous microspheres can be used as functional nanomaterial carriers for their microporous structure and higher specific surface area. In this study, magnetic fluorescent polymer microspheres were prepared by incorporating Fe3O4 nanoparticles and CdSe/ZnS quantum dots(QDs) into hyper-crosslinked microporous polymer microspheres(HCMPs) via the in situ coprecipitation method and swelling-diffusion. The HCMPs predominantly have micropores, and their specific surface area is as high as 703.4 m2/g. The magnetic-fluorescent microspheres maintain the superparamagnetic behavior of Fe3O4, and the saturation magnetization reaches 38.6 A·m2/kg. Moreover, the composite microspheres exhibit an intense emission peak at 530 nm and achieve good fluorescence.

Freezing of water droplet is widely seen and important in the fields of aerospace, cold energy storage, and power production. To investigate the freezing process of a sessile water droplet on a horizontal cold plate, a theoretical model was developed. Different from previously reported models, the effects of supercooling and gravity on the physical properties and the water droplet profile are both considered, respectively. This model is validated with the experimental data of two parameters, including the freezing time and the freezing front radius. The results indicate that the deviation of freezing time is decreased from 7.69% to 0.17%, while the accuracy improved by 7.52%. The average deviation of the freezing front radius is decreased from 142.90 μm to 57.94 μm, with the accuracy improved by 59.46%. At the freezing stage, the appearance of the dynamic growth angle contributes to the less deviation of the freezing front radius. The eccentricity of droplet shape decreases from 0.45 to 0.03, with the eccentricity decreased by 93.45%. The temperature change rates inside a droplet show a gradually decreasing tendency, and the temperatures at different droplet locations present different limiting values. The findings of this study are beneficial for understanding droplet solidification process as well as new technologies for refrigeration, deicing, and defrosting.

In this paper, an impedance control algorithm based on velocity for capturing two low impact docking mechanisms (LIDMs) is presented. The main idea of this algorithm is to track desired forces when the position errors of two LIDMs are random by designing the relationship between the velocity and contact forces measured by a load sensing ring to achieve low impact docking. In this paper, the governing equation of an impedance controller between the deviation of forces and velocity is derived, and simulations are designed to verify how impedance parameters affect the control characteristics. The performance of the presented control algorithm is validated by using the MATLAB and ADAMS software for capturing simulations. The results of capturing simulations demonstrate that the impedance control algorithm can respond fast and has excellent robustness when the environmental errors are random, and the contact forces and torques satisfy the low impact requirements.

The compact and high efficiency coolers working in the liquid nitrogen temperature region play an important role in HTS Applications. Stirling type pulse tube cooler servers as a promising candidate for cooling HTS devices for its advantages such as low vibration, high reliability and low cost due to absence of the moving parts in the cold head compared with traditional coolers. However, phase shift mechanisms used in a conventional pulse tube cryocooler need to dissipate expansion power at the ambient end of the pulse tube, which leads to a lower thermodynamic efficiency than that of a Stirling cryocooler. In order to improve the efficiency and obtain a reliable cryocooler system, this article presents a pulse tube cryocooler which uses room temperature displacers as the phase shifter, which aims at providing more than 10 W cooling power at 77 K. The cryocooler with a model number of TC4189 consists of linear compressor, coaxial pulse tube and two dual-opposed ambient displacers. High pressure ratio and high frequency operation are used to increase the power density. The whole system has a total mass of 4.3 kg. At an optimum working point, a lowest no-load temperature of 44 K has been obtained and the cooling power at 80K reaches 15 W with an input electric power of 240 W, which means an efficiency of 17.1% of Carnot. The influence of displacers operating and structural parameters were investigated through simulations and experiments.

High efficiency cryocoolers working below 10 K have many applications such as cryo-pump, superconductor cooling and cryogenic electronics. This paper presents a thermally coupled two-stage pulse tube cryocooler system and its numeric analysis. The simulation results indicate that temperature distribution in the pulse tube has a significant impact on the system performance. So a precooling heat exchanger is put inside the second stage pulse tube for a deep investigation on its influence on the system performance. The influences of operating parameters such as precooling temperature, location of the precooling heat exchanger are discussed. Comparison of energy losses apparently show the advantages of the configuration which leads to an improvement on the efficiency. Finally, the cryocooler is predicted to be able to reach a relative Carnot efficiency of 10.7% at 10 K temperature.

This thesis presents a study of wireless sensor networks applied in monitoring the curing process of concrete building structure. To replace the massive user intervention of the traditional monitoring methods with automated data acquisition, various types of sensors are developed for different concrete tests, including compressive strength measurement, maturity test, and relative humidity test. The main purpose of the thesis is to provide an analysis of existing commercial sensors as well as research prototypes, comparing different test methods and communication models. Distribution of sensors, future potential in millimeter-wave, and simulation environments are also discussed.

High frequency Stirling type pulse tube cryocooler promises higher specific power which is attractive for many applications. For the past several years, we have been working on a 300 Hz pulse tube cryocooler. At such a high frequency, acoustics plays an important role in determining the cryocooler performance. This paper studies the dependence of cryocooler performance on the dimensions of inertance tube, connecting tube and void volume inside the system from the acoustic viewpoint. While similar influence exists in case of lower frequency systems, the cryocooler are more sensitive to these parameters when the frequency increases up to 300 Hz. Such a study is helpful in guiding the design of the system.

Because lubricating oil for moving parts is not allowed to go into the pulse tube cryocooler, Stirling type pulse tube cryocoolers are generally driven by oil‐free compressors although oil‐lubricated compressors are much cheaper and facile. Recently, it was proposed that an acoustic transparent and oil blocking diaphragm could be employed to separate the compressor and the cryocooler. Thus, the cryocooler can be driven by oil‐lubricated compressors. In this paper, a pulse tube cryocooler is designed to match a crankcase compressor. Although the efficiency of the crankcase compressor is lower compared with the oil‐free linear compressor, the crankcase compressor can easily work at lower frequency which results in higher efficiency for the cryocooler. So the relative high performance of the whole system can be maintained. In this system, the cryocooler delivers 28.5 W of cooling at 80 K with 680 W of electrical input power and operates at 15 Hz. The corresponding Carnot efficiency is 11.52%.

In this paper, a new source video signal is designed for online DVD evaluation and then its composite video signal waveform is produced through a video waveform generator board PXI-5431. The composite video field, no matter NTSC or PAL system it is, includes both visible image and invisible lines. The visible image has been set to different patterns, and the invisible lines include three lines such as copy once, auto clock setting and electronic program guide. Detailed procedures to produce both parts are well illustrated and the virtual instrument blocks in G-language are also presented.

This paper firstly focuses on the video signal measurement system for DVD recorder, one of the most demanding tasks for DVD recorders manufacturing. An online measurement system is configured and set up by PC workstation and LabVIEW, a virtual instrument development platform. After that, the system is put into practice successfully. Some deficient video signals are captured and their image and waveforms are illustrated. The causes behind the deficiency are further analyzed. The online measurement system becomes an essential tool to guarantee the quality of DVD recorders before packing.

During manufacturing DVD recorders, it is necessary to measure the loss between original and play-back video signals, so as to evaluate the output image quality of video channels. This paper presents a local image color components analysis method to analyze the sampled image in a very limited time. A local image area, sized 20times20 pixel, is carefully selected. Its RBG components are calculated and the method is illustrated and coded under the virtual instrument environment LabVIEW and TestStand. Examples are given to demonstrate its feasibility.