Search

The layers of convolutional neural networks (CNNs) can be used to alter the resolution of their inputs, but the scaling factors are limited to integer values. However, in many image and video processing applications, the ability to resize by a fractional factor would be advantageous. One example is conversion between resolutions standardized for video compression, such as from 1080p to 720p. To solve this problem, we propose an alternative building block, formulated as a conventional convolutional layer followed by a differentiable resizer. More concretely, the convolutional layer preserves the resolution of the input, while the resizing operation is fully handled by the resizer. In this way, any CNN architecture can be adapted for non-integer resizing. As an application, we replace the resizing convolutional layer of a modern deep downsampling model by the proposed building block, and apply it to an adaptive bitrate video streaming scenario. Our experimental results show that an improvement in coding efficiency over the conventional Lanczos algorithm is attained, in terms of PSNR, SSIM, and VMAF on test videos., 4 pages conference paper

The use of ℓ [subscript p] norms has largely dominated the measurement of distortion in video encoding or loss in neural networks due to their simplicity and analytical properties. However, when used to assess the loss of visual information, these simple norms are not very consistent with human perception. Given the continuously growing demand for online videos, improving the performance of video compression in perceptual ways has become an important, yet challenging problem, as humans are the ultimate receiver of visual signals. The main contribution of this thesis is to provide new directions for optimization of components in video workflows, in which the topics of hybrid video codecs, resizer, and learned image compression models are covered. The first part of this thesis studies the chroma distortions in conventional video compression standards. It is empirically known that the chroma components are less sensitive to human perception, yet has not been studied as much in the application in video compression. To this end, we carried out a subjective experiment to understand the interplay between luma and chroma distortions. We also found that there is room for reducing bitrate consumption in modern video codecs by creatively increasing the compression factor on chroma channels. On the other hand, video downsampling is also a crucial module in adaptive streaming scenarios. This thesis introduces a new data-driven downsampling model realized using deep neural networks. Since the layers of convolutional neural networks can only be used to alter the resolutions of their inputs by integer scale factors, we seek new ways to achieve fractional scaling, which is crucial in many video processing applications. The second part of this thesis explores the perceptual aspect of optimizing learning-based lossy image compression models. Although numerous powerful perceptual models have been proposed to predict the perceived quality of a distorted picture, most other image quality indexes have never been adopted as deep network loss functions, because they are generally non-differentiable. To address this problem, we propose a new "proximal" approach, called the ProxIQA training, to optimize image analysis networks against quantitative perceptual models. We also describe a search-free resizing framework that can further improve the rate-distortion tradeoff of recent learned image compression models. Our approach is simple: compose a pair of differentiable downsampling/upsampling layers that sandwich a neural compression model. To determine resize factors for different inputs, we utilize another neural network jointly trained with the compression model, with the end goal of minimizing the rate-distortion objective. Among these, quantitative simulations and subjective quality studies show that the proposed methods yield significant improvements in coding efficiency. The thesis concludes with some remarks on future directions and open problems.

This letter proposes the world’s first deep learning (DL)-assisted video encoder LSI fabricated in a 10-nm process with a core area of 0.76 mm2 to integrate quad-core DL accelerators and $4\text{K}\times 2\text{K}$ H.264/H.265 video encoders. A visual-contact-field network (VCFNet) DL model is newly designed to predict human focus information with extraordinary reduction of encoding complexity, leading to 82.3% power reduction. Moreover, input channel reduction and layer merging approaches reduce VCFNet complexity by 69%. Operated at 0.9 V and 504 MHz, the proposed DL-assisted 4K video encoder LSI consumes 56.54 mW to achieve 0.22 nJ/pixel of energy efficiency, cutting 0.1-14 nJ/pixel compared to conventional designs.

This paper proposes the world's first deep learning (DL)-assisted video encoder LSI fabricated in a 10nm process with a core area of 0.76mm2 to integrate quad-core DL accelerators and 4K×2K H.264/H.265 video standards. A visual-contact-field network (VCFNet) DL model is newly designed to predict human focus information for extraordinary reducing the encoding complexity, leading to 82.3% of power reduction. Moreover, input channel reduction and layer merging approaches reduce VCFNet complexity by 69%. Operated at 0.9V and 504MHz, the proposed DL-assisted 4K video encoder LSI consumes 56.54mW to achieve 0.22nJ/pixel of energy efficiency, cutting 0.1-14nJ/pixel compared to conventional designs [1]–[3].

According to the structure characteristics and working modes of a dioxide detection instrument and considering its space environments,a thermal control system for the detection instrument was designed by combination of passive thermal control and active thermal control.Firstly,the structure and the internal heat source were introduced,the heat flux of the instrument was analyzed,and the difficulties of thermal control were obtained.Then,the thermal design of the instrument was carried out and the combination method of passive thermal control and active thermal control was used for the thermal isolation,thermal transmission and the thermal compensation.According to the space environment and thermal control measures,a thermal analysis model was constructed and the thermal transfer was solved with a TMG code.The simulation results show that the temperature of the main frame in the optical system is 13.3 ℃-21.7 ℃,which meets the designrequirements.Finally,the thermal design was verified with a vacuum thermal balance test and the test results indicate that the temperature of the main frame in the optical system is 13.0℃-20.3℃.The tested value is in agreement with the calculated ones,which meets the thermal design targets and verifies that the thermal design is reasonable.

In order to improve the thermal conductivity of materials and decrease the temperature gradient of a space optical remote sensor,the carbon-carbon composites were pasted on the metal surface.The characteristics of carbon-carbon composites were introduced and the heat transfer models of one bare steel plate and two pasted plates with carbon-carbon composites in the thicknesses of 0.5 mm and 2 mm respectively were established.The three heat transfer states of the steel plates were calculated and experimented,and their equivalent thermal conductivities were obtained.The models were simulated by using the finite element model IDEAS-TMG and the heat transfer properties of three kinds of plates mentioned above were compared.Results is shown that the carbon-carbon composites could improve the temperature uniformity of the plates.Finally,the carbon-carbon composites were applied to a TC4 star-tracker bracket of a star sensor,and the temperature difference between the assembly area and the radiator was tested.The tested results indicate that it is 28 ℃ and 5 ℃,respectively,for the bare bracket and the one with 0.5 mm-thick carbon-carbon composites on the inner and outer sides.The tested result illuminates that the thermal conductivity of the bracket of the star-tracker is improved and its temperature gradient is reduced effectively.This method could give some guidances and references for the thermal design of other space optical instruments with great temperature gradients.

A thermal control system for space cameras was designed according to its space environments and structure characteristics.Firstly,the thermal design guidelines of space cameras were summarized,and the thermal environment of a space camera was analyzed.Then,the thermal design of the space camera was carried out.By utilizing the thermal capacitance of a satellite,the passive thermal control was used for thermal isolation and thermal transmission and the active thermal control was conducted to implement the temperature compensation.Finally,four extreme test conditions were designed and thermal balance tests were undertaken according to various work patterns and different thermal environments.The test results show that the temperature difference is 3℃between the space camera and the fitting surface,which meets the system requirements for storage work conditions.Furthermore,the whole space camera temperatures are-3.1℃and 45.7 ℃,the lens temperatures are-4.5℃and 46.8℃in the low temperature and high temperature work conditions,respectively,and they meet the thermal control system requirements.In conclusions,the thermal design of the space camera is feasible and reasonable.

A thermal control system of electric cabinet in a high-resolution space camera was designed to resolve its heat dissipation.The basic flow sheet of thermal design for the electric cabinet was introduced,and several kinds of approaches in thermal conductions and thermal radiation were used to dissipate the heat for the electronic components,Printed Circuit Boards(PCBs) and their chassises.Then,the finite element mode of thermal analysis was corrected according to the thermal test in the vacuum condition.Based on the results of thermal test and thermal analysis, the thermal design was modified by adding a heat pipe additionally.Finally,according to the space environment and structure characteristics,the thermal analysis for the thermal control system was performed by the corrected mode and the case temperatures of the electronic components were calculated.The results show that the case temperatures of electronic components are from 40.2 ℃ to 62.7 ℃ after adding the heat pipe,which means the thermal design scheme is feasible and reasonable for the electric cabinets of space cameras.

A thermal control system for a high-power CCD space camera was designed,including a thermal test of the focal plane assembly for resolving the heat dissipation problem.First,the thermal design scheme of the CCD was examined,where the heat produced by the CCD was transferred to the cold source through a heat pipe.Then a thermal test of focal plane assembly was planned with the simulated cold source specially designed.Finally,the thermal test of focal plane assembly was performed in a vacuum to simulate its space environment and structure characteristics.The results show that the working temperature of a 10 W high-power CCD space camera can be kept below 35 ℃ when it has been working for 70 min continuously and has a 12 ℃ cool source.The thermal design of the CCD focal plane assembly for the space camera is feasible and acceptable.It can be used as a reference for the mode correction of focal plane assembly thermal analysis.