Search

The detection and recognition of traffic signs is an important topic in intelligent transportation systems. The automatic detection and recognition of traffic signs during driving is the basis for realizing the unmanned driving. Therefore, the work on the detection and recognition of traffic signs has a potential value and application prospect. In the traditional detection and recognition methods, they often detect and recognize traffic signs image by image. In this case, only the information of the current image is used, and the relationship between the image sequences is not considered. To end this issue, we propose a novel model that can use the relationship in multi-images to detect and recognize traffic signs in a driving video sequence quickly and accurately. The model proposed in this paper is a fusion model based on YOLO-V3 and VGG19 network. Finally, we test this proposed model on a public dataset and compare it to the baseline method, and results show that this proposed model achieves accuracy over 90% and outperforms the baseline method for all types of traffic signs in different conditions. Thus, we can conclude this proposed model is efficient and accurate. [ABSTRACT FROM AUTHOR]

Traffic sign recognition (TSR) plays an important role in driving assistance system and traffic safety insurance. However, existing methods focus on extracting features of traffic signs and ignore the constraints of spatial positional relationships between traffic signs and other objects in the scene. This way results in incorrectly detecting other similar objects as traffic signs and failing to detect very small traffic signs. A TSR method based on semantic scene understanding and structural traffic sign location is proposed in this study to solve the aforementioned problems. A scene structure model based on the constraints of spatial positional relationships between traffic signs and other objects is proposed to establish trusted search regions. An improved Light-weight RefineNet is used to analyze and understand a scene semantically and accurately and then segment objects in complicated environments precisely. A new network multiscale densely connected object detector (MDCOD) based on densely connected style, multiscale feature fusion, and improved K-means++ algorithms is proposed to recognize very small traffic signs. The trusted traffic signs are found by filtering false candidates outside the scene structure model. The proposed method is tested on Tsinghua-Tencent 100K and German Traffic Sign Detection Benchmark datasets and achieves accuracies of 92.8% and 99.90%, respectively, outperforming the existing methods. [ABSTRACT FROM AUTHOR]

Vehicle re-identification (Re-ID) is one of the promising applications in the field of computer vision. Existing vehicle Re-ID methods mainly focus on global appearance features or pre-defined local region features, which have difficulties in handling inter-class similarities and intra-class differences among vehicles in various traffic scenarios. This paper proposes a novel end-to-end three-branch embedding network (TBE-Net) with feature complementary learning and part-aware ability. The proposed TBE-Net integrates complementary features, global appearance, and local region features into a unified framework for subtle feature learning, thereby obtaining more integral and diverse vehicle features to re-identify the vehicle from similar ones. The local region feature branch in the proposed TBE-Net contains an attention module that highlights the major differences among local regions by adaptively assigning large weights to the critical local regions and small weights to insignificant local regions, thereby enhancing the perception sensitivity of the network to subtle discrepancies. The complementary branch in the proposed TBE-Net exploits different pooling operations to obtain more comprehensive structural features and multi-granularity features as a supplement to the global appearance and local region features. The abundant features help accommodate the ever-changing critical local regions in vehicles’ images due to the sensors’ settings, such as the position and shooting angle of surveillance cameras. The extensive experiments on VehicleID and VeRi-776 datasets show that the proposed TBE-Net outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

We analyzed 23 mining pools and explore the mobility of miners throughout Bitcoin’s history. Mining pools have emerged as major players to ensure that the Bitcoin system stays secure, valid, and stable. Many questions remain open regarding how mining pools have evolved throughout Bitcoin’s history and when and why miners join or leave the pools. We investigated the reward payout flow of mining pools and characterized them based on payout irregularity and structural complexity. Based on our proposed algorithm, we identified miners and studied their mobility in the pools over time. Our analysis shows that Bitcoin mining is an industry that is sensitive to external events (e.g., market price and government policy). Over time, competition between pools involving reward schemes and pool fees motivated miners to migrate between pools (i.e., pool hopping and cross pooling). These factors converged toward optimal scheme and values, which made mining activities more stable. [ABSTRACT FROM AUTHOR]

This study aimed to evaluate the feasibility of using the PhotoMetrix PRO® application to perform the selection of good quality corn seeds from defective ones, through digital images and the multivariate analysis of color elements. An optimization of the experimental conditions was carried out. Eighteen experiments were performed varying the size of the analysis recipient, the distance of the sample from the smartphone and the size of the region of interest (ROI). A calibration model was built using a partial least squares (PLS) regression model in the determined experimental conditions, and the prediction of external samples was then performed. The conditions that provided the smallest cross‐validation error consisted of using the smaller chamber, the largest ROI, and the longest distance from the smartphone to the sample. The coefficient of determination obtained by the model was.976. In addition, the root‐mean‐square error of calibration (RMSEC) and the root‐mean‐square error of cross‐validation (RMSECV) obtained were 1.2% and 4.8%, respectively. In the prediction, two outliers were observed, and the RMSEP dropped to 2.2% when they were removed. The proposed methodology shows great potential to contribute to the industrial process of corn seed production. It is fast, accurate, and low cost, allowing the analysis to be performed 15 times faster than the current process. Practical Applications: Corn is one of the most important commodities in Brazil's economy. During the processing of corn seeds in the industry, those of good quality are selected among the others through parameters, such as color and density, for example. The gravity table and the Color Sorter machine are equipment used for this purpose. However, even if the machine's separation is effective, an important percentage of good seeds are wasted. To reduce the waste of good seeds remaining in the tailings of these machines, the operators carry out a manual inspection of tailings samples, determining the quantity of quality seeds present there for later adjustment of the machines. This visual and manual analysis becomes time‐consuming, subjective, and imprecise. Thus, as an alternative, this study proposes the use of digital image and partial least square model built by a mobile app to determine the quantity of quality corn seeds in the machines' wastes. [ABSTRACT FROM AUTHOR]

Malachite green (MG) is a triphenylmethane cationic dye used in aquaculture practice, although it has been banned in several countries. The illegal use by fish producers, however, persists due to its effectiveness, and ready and cheap supply. To prevent indiscriminate applications, strict control measures with simple analytical approaches are therefore necessary. With this purpose, a novel, cheap and simple method applying electrospun polyimide nanofibers was developed and validated for MG control in water by color image analysis. For detection, a simple apparatus and ImageJ® software to treat images captured by common smartphones were used. A detection limit of 0.013 mg/L with a linear analytical response range within the concentration of 0.05 to 0.3 mg/ L of malachite green (MG) with a correlation coefficient of 0.997 and standard deviation (n = 9) varying from 1.01 to 3.92% was achieved with the proposed method. Accuracy was assessed by recovery assays in water samples and percentages of 96.6 to 102.0% were obtained. The method is robust and suitable for the rapid and reliable monitoring of MG in water. [ABSTRACT FROM AUTHOR]

Compared to element-wise embedding, vector-wise embedding based on CPV (color pixel vector) shows its superiority in color image steganography. However, when working with an adversarial embedding scheme for introducing adversarial perturbations, its success rate of deceiving a target CNN (convolutional neural network) steganalyzer dramatically drops. In this paper, inspired by the I-FGSM (iterative fast gradient sign method), we present an effective steganography for color images. Specifically, after decomposing an image into several non-overlapped sub-images, we iteratively and gradually increase the possibilities of generating adversarial perturbations for the CPVs in each sub-image by changing their adversarial costs. The costs are incrementally adjusted with a small step so that their maximum relative variation is minimized. Leveraging a new designed cost adjustment criterion, more modification patterns of CPV can participate in producing effective adversarial perturbations. Extensive experiments demonstrate that the proposed method achieves a high success rate in deceiving the target CNN steganalyzer and stably defending against the detection of other non-target steganalytic schemes for color images. [ABSTRACT FROM AUTHOR]

Images acquired in low-light conditions suffer from a series of visual quality degradations, e.g., low visibility, degraded contrast, and intensive noise. These complicated degradations based on various contexts (e.g., noise in smooth regions, over-exposure in well-exposed regions and low contrast around edges) cast major challenges to the low-light image enhancement. Herein, we propose a new methodology by imposing a learnable guidance map from the signal and deep priors, making the deep neural network adaptively enhance low-light images in a region-dependent manner. The enhancement capability of the learnable guidance map is further exploited with the multi-scale dilated context collaboration, leading to contextually enriched feature representations extracted by the model with various receptive fields. Through assimilating the intrinsic perceptual information from the learned guidance map, richer and more realistic textures are generated. Extensive experiments on real low-light images demonstrate the effectiveness of our method, which delivers superior results quantitatively and qualitatively. The code is available at https://github.com/lingyzhu0101/GEMSC to facilitate future research. [ABSTRACT FROM AUTHOR]

Deep learning networks have recently demonstrated yielded impressive progress for multi-exposure image fusion. However, how to restore realistic texture details while correcting color distortion is still a challenging problem to be solved. To alleviate the aforementioned issues, in this paper, we propose an attention-guided global-local adversarial learning network for fusing extreme exposure images in a coarse-to-fine manner. Firstly, the coarse fusion result is generated under the guidance of attention weight maps, which acquires the essential region of interest from both sides. Secondly, we formulate an edge loss function, along with a spatial feature transform layer, for refining the fusion process. So that it can take full use of the edge information to deal with blurry edges. Moreover, by incorporating global-local learning, our method can balance pixel intensity distribution and correct the color distortion on spatially varying source images from both image/patch perspectives. Such a global-local discriminator ensures all the local patches of the fused images align with realistic normal-exposure ones. Extensive experimental results on two publicly available datasets show that our method drastically outperforms state-of-the-art methods in visual inspection and objective analysis. Furthermore, sufficient ablation experiments prove that our method has significant advantages in generating high-quality fused results with appealing details, clear targets, and faithful color. Source code will be available at https://github.com/JinyuanLiu-CV/AGAL. [ABSTRACT FROM AUTHOR]

With the rapid development of screen content video applications, screen content coding (SCC) is urgently needed to be used in commercial codecs. However, the extra encoding complexity introduced by the new SCC tools has posed a great challenge for its practical deployment. In this paper, motivated by our observations that there should be a fine-grained mapping between image content and candidate modes, we propose a two-step fast mode decision method to reduce the encoding complexity. First, we propose to use a convolution neural network (CNN) to automatically extract useful features for fine-grained content classification. Second, we build a precise and concise mapping from CUs to candidate modes by simultaneously considering CU content type, CU size, and mode complexity. Note that the spatial correlations between neighboring CUs and current CU are also utilized in candidate modes derivation. In addition to the two-step fast mode decision method, a content-aware early termination algorithm is further proposed to reduce the encoding complexity. Extensive experiments demonstrate that our method achieves better performance compared with state-of-the-art ones, with 50.13% total encoding complexity reduction and only 0.92% BD-rate increase. [ABSTRACT FROM AUTHOR]

Global structure and local detailed texture have different effects on image enhancement tasks. However, most existing works treated these two components in the same way, without fully considering the characteristics of the global structure and local detailed texture. In this work, we propose a structure-texture aware network (STANet) that successfully exploits structure and texture features of low-light images to improve perceptual quality. To construct STANet, a fine-scale contour map guided filter is introduced to decompose the image into a structure component and a texture component. Then, structure-attention and texture-attention subnetworks are designed to fully exploit the characteristics of these two components. Finally, a fusion subnetwork with attention mechanisms is utilized to explore the internal correlations among the global and local features. Furthermore, to optimize the proposed STANet model, we propose a hybrid loss function; specifically, a color loss function is introduced to alleviate color distortion in the enhanced image. Extensive experiments demonstrate that the proposed method improves the visual quality of images; moreover, STANet outperforms most other state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Text-to-image synthesis is a challenging task that generates realistic images from a textual description. The description contains limited information compared with the corresponding image and is ambiguous and abstract, which will complicate the generation and lead to low-quality images. To address this problem, we propose a novel generation text-to-image synthesis method, called RiFeGAN2, to enrich the given description. To improve the enrichment quality while accelerating the enrichment process, RiFeGAN2 exploits a domain-specific constrained model to limit the search scope and then uses an attention-based caption matching model to refine the compatible candidate captions based on constrained prior knowledge. To improve the semantic consistency between the given description and the synthesized results, RiFeGAN2 employs improved SAEMs, SAEM2s, to compact better features of the retrieved captions and effectively emphasize the descriptions via incorporating centre-attention layers. Finally, multi-caption attentional GANs are exploited to synthesize images from those features. Experiments performed on widely-used datasets show that the models can generate vivid images from enriched captions and effectually improve the semantic consistency. [ABSTRACT FROM AUTHOR]

This paper presents a novel method, Zero-Reference Deep Curve Estimation (Zero-DCE), which formulates light enhancement as a task of image-specific curve estimation with a deep network. Our method trains a lightweight deep network, DCE-Net, to estimate pixel-wise and high-order curves for dynamic range adjustment of a given image. The curve estimation is specially designed, considering pixel value range, monotonicity, and differentiability. Zero-DCE is appealing in its relaxed assumption on reference images, i.e., it does not require any paired or even unpaired data during training. This is achieved through a set of carefully formulated non-reference loss functions, which implicitly measure the enhancement quality and drive the learning of the network. Despite its simplicity, we show that it generalizes well to diverse lighting conditions. Our method is efficient as image enhancement can be achieved by an intuitive and simple nonlinear curve mapping. We further present an accelerated and light version of Zero-DCE, called Zero-DCE++, that takes advantage of a tiny network with just 10K parameters. Zero-DCE++ has a fast inference speed (1000/11 FPS on a single GPU/CPU for an image of size 1200×900×3) while keeping the enhancement performance of Zero-DCE. Extensive experiments on various benchmarks demonstrate the advantages of our method over state-of-the-art methods qualitatively and quantitatively. Furthermore, the potential benefits of our method to face detection in the dark are discussed. The source code is made publicly available at https://li-chongyi.github.io/Proj_Zero-DCE++.html. [ABSTRACT FROM AUTHOR]

Manual inspection of baggage X-ray images for the detection of threat materials such as explosives is a tedious and stressful task. We have presented a convolutional neural network (CNN)-based approach for automated detection of threat materials in X-ray images. A deep convolutional generative adversarial network (DCGAN)-based data synthesized model is used for the generation of datasets containing gray images (DXBD1) and pseudo-colored images (DXBD2). A deep CNN-based model, named dual X-ray Baggage (DXB), is proposed for automated classification of threat materials in baggage X-ray images. The proposed model consists of seven layers comprising two convolution layers and five fully connected layers. With this model, accuracies of 98% for gray images with F1 score of 0.97 and 99% for pseudo-colored images with F1 score of 0.99 have been achieved. The proposed model requires the classification time of about 12 s and 6 s, respectively, for gray and pseudo-colored images indicating its suitability for real-time monitoring. [ABSTRACT FROM AUTHOR]

Contribution: A new operational definition of series connections is given based on elements belonging to the same two meshes, which is properly dual to the usual definition of parallel elements being connected to the same two nodes. Furthermore, computer-based exercises have been developed and tested to teach students about such connections in gateway linear circuits courses, using color coding of nodes and meshes as a pedagogical device. Background: Series and parallel connections are a crucial but difficult concept. Existing textbooks give them limited attention, resulting in later difficulties learning circuit analysis. Research Questions: RQ1: Can an improved definition of series elements aid student understanding and student satisfaction? RQ2: Can a computer-based “game” lead to effective mastery and student satisfaction at a wide range of institutions, including minority-serving ones? Methodology: Standard and new definitions were elaborated in a multiple-choice tutorial. A game was developed focusing on identifying series and parallel connections, with color coding of both nodes and meshes. Student learning was assessed over eight years using pretest and posttest in 14 varied institutions. Student opinions were assessed using several types of surveys. Findings: Strong learning gains were observed every semester from built-in pretest and posttest, with average scores of 28% and 87%, respectively. Large improvements were observed at every institution including five minority-serving ones. The posttest score is increased by a statistically significant amount after introducing the new definition of series elements. Students preferred the new definition of series and recommended its use, and very strongly endorsed color coding. [ABSTRACT FROM AUTHOR]

Existing shadow removal methods often struggle with two problems: color inconsistencies in shadow areas and artifacts along shadow boundaries. To address these two problems, we propose a novel shadow mask-based semantic-aware network (S2Net) that uses shadow masks as guidance for shadow removal. The color inconsistency problem is solved in two steps. First, we use a series of semantic-guided dilated residual (SDR) blocks to transfer statistical information from non-shadow areas to shadow areas. The shadow mask-based semantic transformation (SST) operation in SDR enables the network to remove shadows while keeping non-shadow areas intact. Then, we design a refinement block by incorporating semantic knowledge of shadow masks and applying the learned modulated convolution kernels to get traceless and consistent output. To remove artifacts along shadow boundaries, we propose a newly designed boundary loss. The boundary loss encourages spatial coherence around shadow boundaries. By including the boundary loss as part of the loss function, a significant portion of artifacts along shadow boundaries can be removed. Extensive experiments on the ISTD, ISTD+, SRD and SBU datasets show our S2Net outperforms existing shadow removal methods. [ABSTRACT FROM AUTHOR]

Lane detection is essential for autonomous vehicles, and vision-based lane detection is widely used in the field of intelligent driving cars because of its low cost. Aiming to solve the problem of false detection caused by surrounding vehicles during lane detection, a new multi-lane detection method with strong anti-interference ability is proposed. The main contribution of this work is the utilization of object detection neural network which is applied to remove vehicles from road images as much as possible. Compared with the traditional feature-based detection algorithms, this method can preserve lane line features more accurately and effectively. Firstly, binarization of global optimal threshold method is utilized to extract the basic features of lanes. In order to further remove non-lane line noises and extract lane features as clear as possible, YOLOv4-tiny object detection network is introduced to detect and filter vehicles on the road. Then the DBSCAN (Density-Based Spatial Clustering of Applications with Noise) algorithm with denoising function is applied to cluster the different lane features. Finally, an improved RANSAC (Random Sample Consensus) method is used to perform quadratic curve fitting on the clustered features. The experimental results on dataset show that consumed time of a single frame image is only 0.119 second and the precision of the proposed method can reach 94.4%. Compared with other detection algorithms, the proposed algorithm has good real-time performance, stability and robustness in a variety of complex scenarios. [ABSTRACT FROM AUTHOR]

Recent work in data visualization has demonstrated that small, perceptually-distinct color palettes—such as those used in categorical mapping—can connote significant affective qualities. Data that are mapped or otherwise visualized are also often emotive in nature, either inherently (e.g., climate change, disease mortality rates), or by design, such as can be found in visual storytelling. However, little is known about how the affective qualities of color interact with those of data context in visualization design. This article describes the results of a crowdsourced study on the influence of affectively congruent versus incongruent color schemes on categorical map-reading response. We report both objective (pattern detection; area comparison) and subjective (affective quality; appropriateness; preference) measures of map-reader response. Our results suggest that affectively congruent colors amplify perceptions of the affective qualities of maps with emotive topics, affective incongruence may cause confusion, and that affective congruence is particularly influential in maps of positive-leaning data topics. Finally, we offer preliminary design recommendations for balancing color congruence with other design factors, and for synthesizing color and affective context in thematic map design. [ABSTRACT FROM AUTHOR]

We propose a novel framework to produce cartoon videos by fetching the color information from two input keyframes while following the animated motion guided by a user sketch. The key idea of the proposed approach is to estimate the dense cross-domain correspondence between the sketch and cartoon video frames, and employ a blending module with occlusion estimation to synthesize the middle frame guided by the sketch. After that, the input frames and the synthetic frame equipped with established correspondence are fed into an arbitrary-time frame interpolation pipeline to generate and refine additional inbetween frames. Finally, a module to preserve temporal consistency is employed. Compared to common frame interpolation methods, our approach can address frames with relatively large motion and also has the flexibility to enable users to control the generated video sequences by editing the sketch guidance. By explicitly considering the correspondence between frames and the sketch, we can achieve higher quality results than other image synthesis methods. Our results show that our system generalizes well to different movie frames, achieving better results than existing solutions. [ABSTRACT FROM AUTHOR]

We present GridSet, a novel set visualization for exploring elements, their attributes, intersections, as well as entire sets. In this set visualization, each set representation is composed of glyphs, which represent individual elements and their attributes utilizing different visual encodings. In each set, elements are organized within a grid treemap layout that can provide space-efficient overviews of the elements structured by set intersections across multiple sets. These intersecting elements can be connected among sets through visual links. These visual representations for the individual set, elements, and intersection in GridSet facilitate novel interaction approaches for undertaking analysis tasks by utilizing both macroscopic views of sets, as well as microscopic views of elements and attribute details. In order to perform multiple set operations, GridSet supports a simple and straightforward process for set operations through dragging and dropping set objects. Our use cases involving two large set-typed datasets demonstrate that GridSet facilitates the exploration and identification of meaningful patterns and distributions of elements with respect to attributes and set intersections for solving complex analysis problems in set-typed data. [ABSTRACT FROM AUTHOR]

Augmented reality (AR) offers new ways to visualize information on-the-go. As noted in related work, AR graphics presented via optical see-through AR displays are particularly prone to color blending, whereby intended graphic colors may be perceptually altered by real-world backgrounds, ultimately degrading usability. This work adds to this body of knowledge by presenting a methodology for assessing AR interface color robustness, as quantitatively measured via shifts in the CIE color space, and qualitatively assessed in terms of users’ perceived color name. We conducted a human factors study where twelve participants examined eight AR colors atop three real-world backgrounds as viewed through an in-vehicle AR head-up display (HUD); a type of optical see-through display used to project driving-related information atop the forward-looking road scene. Participants completed visual search tasks, matched the perceived AR HUD color against the WCS color palette, and verbally named the perceived color. We present analysis that suggests blue, green, and yellow AR colors are relatively robust, while red and brown are not, and discuss the impact of chromaticity shift and dispersion on outdoor AR interface design. While this work presents a case study in transportation, the methodology is applicable to a wide range of AR displays in many application domains and settings. [ABSTRACT FROM AUTHOR]

Recoloring 3D models is a challenging task that often requires professional knowledge and tedious manual efforts. In this article, we present the first deep-learning framework for exemplar-based 3D model recolor, which can automatically transfer the colors from a reference image to the 3D model texture. Our framework consists of two modules to solve two major challenges in the 3D color transfer. First, we propose a new feed-forward Color Transfer Network to achieve high-quality semantic-level color transfer by finding dense semantic correspondences between images. Second, considering 3D model constraints such as UV mapping, we design a novel 3D Texture Optimization Module which can generate a seamless and coherent texture by combining color transferred results rendered in multiple views. Experiments show that our method performs robustly and generalizes well to various kinds of models. [ABSTRACT FROM AUTHOR]

Color design for 3D indoor scenes is a challenging problem due to many factors that need to be balanced. Although learning from images is a commonly adopted strategy, this strategy may be more suitable for natural scenes in which objects tend to have relatively fixed colors. For interior scenes consisting mostly of man-made objects, creative yet reasonable color assignments are expected. We propose $C^{3}$ C 3 Assignment, a system providing diverse suggestions for interior color design while satisfying general global and local rules including color compatibility, color mood, contrast, and user preference. We extend these constraints from the image domain to $\mathbb {R}^3$ R 3 , and formulate 3D interior color design as an optimization problem. The design is accomplished in an omnidirectional manner to ensure a comfortable experience when the inhabitant observes the interior scene from possible positions and directions. We design a surrogate-assisted evolutionary algorithm to efficiently solve the highly nonlinear optimization problem for interactive applications, and investigate the system performance concerning problem complexity, solver convergence, and suggestion diversity. Preliminary user studies have been conducted to validate the rule extension from 2D to 3D and to verify system usability. [ABSTRACT FROM AUTHOR]

Visual surveillance has become indispensable in the evolution of Intelligent Transportation Systems (ITS). Video object trajectories are key to many of the visual surveillance applications. Classifying varying length time series data such as video object trajectories using conventional neural networks, can be challenging. In this paper, we propose trajectory classification and anomaly detection using a hybrid Convolutional Neural Network (CNN) and Variational Autoencoder (VAE) architecture. First, we introduce a high level features for varying length object trajectories using color gradient representation. In the next stage, a semi-supervised way to annotate moving object trajectories extracted using Temporally Incremental Gravitational Model (TIGM) is used for class labeling. For training, anomalous trajectories are identified using t-Distributed Stochastic Neighbor Embedding (t-SNE). Finally, a hybrid CNN-VAE architecture has been proposed for trajectory classification and anomaly detection. The results obtained using publicly available surveillance video datasets reveal that the proposed method can successfully identify traffic anomalies such as violations in lane driving, sudden speed variations, abrupt termination of vehicle during movement, and vehicles moving in wrong directions. The accuracy of trajectory classification improves by a margin of 1-6% against popular neural networks-based classifiers across various datasets using the proposed high-level features. The gradient representation also improves the anomaly detection accuracy significantly (30-35%). Code and dataset can be found at https://github.com/santhoshkelathodi/CNN-VAE. [ABSTRACT FROM AUTHOR]

Multi-object tracking involves maintaining several trajectories of different objects moving in the scene throughout the video. With this objective, in this article, a fully automatic and real-time tracking algorithm to track multiple vehicles in a video is proposed. The proposed method specifically tries to address the challenges of occlusion and fast-motion in traffic surveillance. The algorithm begins with automatic detection of the moving targets by an adaptive GMM-based background subtraction method. The trajectories of these detected targets are then built using a three-level multi-motion modeled particle filter framework which allows to deal with the challenges of occlusion and fast-motions of the target. The likelihood model for targets is based on their color distribution and edge oriented histogram features. It is contended that the color distribution feature, which can represent the target appearance and the edge oriented histogram, which can describe the target structure are sufficient to represent it in an unique feature space. Based on the similarity of target likelihood, the locations of the targets are filtered. These filtered locations are then associated with the most likely detections using the proposed low-cost and fast data association algorithm based on Euclidean distance and prevailing motion vector. The performance evaluation of the proposed scheme is carried out based on the six measures: Multi Object Tracking Accuracy, Multi Object Tracking Precision, Mostly Tracked trajectories, Mostly Lost trajectories, Identity Switches and Frames Per Second. The results evaluated on stationary camera shot sequences from benchmark datasets as well as real-time shot videos indicate that the proposed algorithm ensures robust tracking and can be used effectively for real-time surveillance of highways. [ABSTRACT FROM AUTHOR]

Wavelet transform is being widely used in classical image processing. One-dimension quantum wavelet transforms (QWTs) have been proposed. Generalizations of the 1-D QWT into multilevel and multidimension have been investigated but restricted to the quantum wavelet packet transform (QWPTs), which is the direct product of 1-D QWPTs, and there is no transform between the packets in different dimensions. A 2-D QWT is vital for image processing. We construct the multilevel 2-D QWT’s general theory. Explicitly, we built multilevel 2-D Haar QWT and the multilevel Daubechies D4 QWT, respectively. We have given the complete quantum circuits for these wavelet transforms, using both noniterative and iterative methods. Compared to the 1-D QWT and wavelet packet transform, the multilevel 2-D QWT involves the entanglement between components in different degrees. Complexity analysis reveals that the proposed transforms offer exponential speedup over their classical counterparts. Also, the proposed wavelet transforms are used to realize quantum image compression. Simulation results demonstrate that the proposed wavelet transforms are significant and obtain the same results as their classical counterparts with an exponential speedup. [ABSTRACT FROM AUTHOR]

Feature extraction and matching are the basis of many computer vision problems, such as image retrieval, object recognition, and visual odometry. In this article, we present a novel RGB-D feature with texture and geometric information (TG). It consists of a keypoint detector and a feature descriptor, which is accurate, efficient, and robust to scene variance. In the keypoint detection, we build a simplified Gaussian image pyramid to extract the texture feature. Meanwhile, the gradient of the point cloud is superimposed as the geometric feature. In the feature description, the texture information and spatial information are encoded in relative order to build a discriminative descriptor. We also construct a novel RGB-D benchmark dataset for RGB-D detector and descriptor evaluation under single variation. Comprehensive experiments are carried out to prove the superior performance of the proposed feature compared with state-of-the-art algorithms. The experimental results also demonstrate that our TG can achieve better performance especially on accuracy and the computational efficiency, making it more suitable for the real-time applications, e.g., visual odometry. [ABSTRACT FROM AUTHOR]

Fast-Match Algorithm is a fast and randomized template matching algorithm based on affine transformations. However, it is not suitable for color images as they are converted into greyscale images, and hence the color information is lost. An improved algorithm named Color Fast Affine Template Matching (CFAsT-Match), was thus proposed which utilizes all the three color channels (R, G, B). Our previous works have demonstrated how CFAsT-Match can be effectively used in shoe detection problem, wherein a robot can successfully track a target person from consecutive frames captured by an onboard camera. This paper proposes a new technique which extends the template matching algorithm using the Fuzzy Density based Clustering Algorithm instead of the traditional DBSCAN Algorithm. Three variants of Fuzzy based DBSCAN algorithms have been implemented here and a novel similarity measure called Fuzzy-based-colour-sum-of-absolute-differences (FCSAD) has been proposed. Experimental results firmly demonstrate that Fuzzy DBSCAN based CFAsT-Match algorithms can comfortably outperform recently proposed crisp DBSCAN based CFAsT-Match algorithms for shoe detection in human-robot coexisting environments. [ABSTRACT FROM AUTHOR]

In the era of artificial intelligence perceptual algorithms used in state-of-the-art Advanced Driver Assistance Systems (ADAS), algorithm validation is not an easy task. To ensure the highest possible safety level of the solution, the performance of the algorithm must be evaluated under a variety of challenging conditions. To test the algorithms, simulators are used to emulate the virtual environment around the car taking into account road traffic, infrastructure and vehicles dynamics. Sensor models are necessary for virtual testing to provide required data to ADAS algorithms. This article introduces the issue of modeling the color filter spaces that are used in the automotive industry. The images generated by the simulator usually have RGB color. In contrast, the automotive industry uses filters such as RCCC and RYYCy. In this paper, the methods for transforming color space from RGB to RYYCy are discussed. Three novel approaches are introduced to solve this problem: analytical, polynomial, and based on a neural network. Moreover, comparative discussion of the presented solutions is shown and with the set of experiments the conversion accuracy and execution time of each algorithm are compared. In addition, introduced solution were compared with modified models that are presented in the literature. [ABSTRACT FROM AUTHOR]

The dynamical analysis, field programmable gate array (FPGA) implementation and image encryption of a linear piecewiseresistive-capacitive-inductive shunted Josephson junction (LPRCISJJ) model are reported in this paper. The numerical simulations of LPRCISJJ model reveal the existence of periodic attractors, bistable periodic attractors, antimonotonicity phenomenon and hidden chaotic attractors with different shapes by varying its parameters. Numerical simulations and FPGA results produce alike timing evolutions and phase planes. Chaotic LPRCISJJ model is used as a pseudo-random number generator (PRNG) in the design of cryptography algorithm based on bit-level permutation. The designed algorithm performs bit masking and relocation depending on the amount of information the bit contains in the pixel. The standard image security analysis on color Lena image of size 512 × 512 are performed and compared with good standing papers in the literature. Prominent outcomes in term of mean entropy of 7.9994, mean NPCR of 99.62%, key sensitivity percentage of 99.68%, average correlation of 0.002 are obtained just to name a few. These security tests carried out certify its resistance to common attacks and its high sensitivity to one-bit changes in the key or image to be encrypted. [ABSTRACT FROM AUTHOR]

Video compression is an important procedure for digital applications. A commonly used format for inputs to video coders is YUV420, which is a subsampled result from YUV444, that is transformed and demosaicked from a color filter array (CFA) format. The process of chroma subsampling is a crucial step for the reconstructed image quality. A state-of-the-art method utilizes the solution of a prior method to compute the starting point, and devises a search method to improve the results. However, the search method does not consider the optimality condition of the problem, and the overall computational complexity is high because of the execution of the prior method for the starting point. In the proposed work, the cost function of the optimization problem is analyzed and decomposed into 4 subchannels based on the CFA format. The optimal line of each subterm is studied in 3-dimensional space. The problem is then mathematically reduced to a 3-subchannel problem. Based on the combinations of the optimal lines, a triangle search area is formed and proved to contain the optimal solution. And because the derived triangle search area is small on average, the search for the optimal value can be considerably fast. Experimental results demonstrate that the proposed approach can produce reconstructed images with qualities that are exactly the same as those generated using the state-of-the-art method. Relative to the state-of-the-art method, the proposed approach can reduce the time complexity by 84.08%–84.91%, and reduce the number of search points by 40.24%–85.25%, for the image datasets Kodak and IMAX with different demosaicking methods. [ABSTRACT FROM AUTHOR]

Spatially varying exposure (SVE) is a promising choice for high-dynamic-range (HDR) imaging (HDRI). The SVE-based HDRI, which is called single-shot HDRI, is an efficient solution to avoid ghosting artifacts. However, it is very challenging to restore a full-resolution HDR image from a real-world image with SVE because: a) only one-third of pixels with varying exposures are captured by camera in a Bayer pattern, b) some of the captured pixels are over- and under-exposed. For the former challenge, a spatially varying convolution (SVC) is designed to process the Bayer images carried with varying exposures. For the latter one, an exposure-guidance method is proposed against the interference from over- and under-exposed pixels. Finally, a joint demosaicing and HDRI deep learning framework is formalized to include the two novel components and to realize an end-to-end single-shot HDRI. Experiments indicate that the proposed end-to-end framework avoids the problem of cumulative errors and surpasses the related state-of-the-art methods. Related codes and datasets will be provided at https://github.com/yilun-xu/SVEHDRI/. [ABSTRACT FROM AUTHOR]

As two practical and important image processing tasks, color demosaicking (CDM) and super-resolution (SR) have been studied for decades. However, most literature studies these two tasks independently, ignoring the potential benefits of a joint solution. In this paper, aiming at efficient and effective joint demosaicking and super-resolution (JDSR), a well-designed two-stage convolutional neural network (CNN) architecture is proposed. For the first stage, by making use of the sampling-pattern information, a pattern-aware feature extraction (PFE) module extracts features directly from the Bayer-sampled low-resolution (LR) image, while keeping the resolution of the extracted features the same as the input. For the second stage, a dual-branch feature refinement (DFR) module effectively decomposes the features into two components with different spatial frequencies, on which different learning strategies are applied. On each branch of the DFR module, the feature refinement unit, namely, densely-connected dual-path enhancement blocks (DDEB), establishes a sophisticated nonlinear mapping from the LR space to the high-resolution (HR) space. To achieve strong representational power, two paths of transformations and the channel attention mechanism are adopted in DDEB. Extensive experiments demonstrate that the proposed method is superior to the sequential combination of state-of-the-art (SOTA) CDM and SR methods. Moreover, with much smaller model size, our approach also surpasses other SOTA JDSR methods. [ABSTRACT FROM AUTHOR]

Most of the existing blind omnidirectional image quality assessment (BOIQA) methods are based on data-driven approach where the end-to-end neural network or deep learning tools are mainly used for feature extraction. However, it usually lacks interpretability and is difficult to discover the perceptual mechanism behind. In this paper, from the perspective of perception modeling, we propose a novel multi-angle projection based BOIQA (MP-BOIQA) method. Considering the omnibearing and near eye display characteristics with head mounted display, multiple color cubemap projection images with respect to different viewpoints are grouped as the color omnidirectional distortion (COD) units so as to simulate the user’s viewing behavior in subjective quality assessment. In the designed multi-angle projection based feature extractor, tensor decomposition is implemented on each COD unit for dimensionality reduction, and piecewise exponential fitting is used to get the distribution of mean subtracted contrast normalized coefficients of the unit’s feature matrices in tensor domain. Finally, the extracted features are pooled with random forest. The experimental results on three omnidirectional image quality datasets show that the MP-BOIQA method can deliver highly competitive performance compared with some representative full-reference quality assessment methods, as well as some state-of-the-art BOIQA methods. [ABSTRACT FROM AUTHOR]

In this letter, a robust and promising atmospheric scattering model (ASM)-based image dehazing technique called IDBP is developed, which overcomes the intrinsic limitation of available techniques based on single priors. It consists of two modules, i.e., an atmospheric light estimation (ALE) module and a multiple prior constraint (MPC) module. The ALE module is based on a new global brightening strategy of enhancing the brightness of image with minimum information loss. The MPC smartly blends the constrains of non-local prior, local prior, and global prior to shrink the solution space of haze removal, which avoids the limitation of using any single priors. Unlike previous works, IDBP does not require any training process, but is based on multiple priors and minimal information loss principle to impose the ASM, thereby making it easy to implement and ensuring its robustness. Numerous experiments reveal that the proposed IDBP outperforms the state-of-the-art alternates. [ABSTRACT FROM AUTHOR]

This letter introduces an underwater image enhancement method to handle low contrast and color cast of underwater images. Firstly, with the help of hierarchical searching technique, we propose a novel backscattered light estimation method. And in this procedure, a novel scoring formula is considered into our method, which comprehensively considers multiple prior knowledge. Then, we generalize underwater dark channel prior (UDCP) approach to obtain more robust transmission estimation. In addition, we also develop a white balance method to further modify the appearance of the resultant image. Extensive experiments on real-world images demonstrate that the proposed method outperforms several previous image restoration or enhancement works. [ABSTRACT FROM AUTHOR]

This paper proposes a novel pretext task to address the self-supervised video representation learning problem. Specifically, given an unlabeled video clip, we compute a series of spatio-temporal statistical summaries, such as the spatial location and dominant direction of the largest motion, the spatial location and dominant color of the largest color diversity along the temporal axis, etc. Then a neural network is built and trained to yield the statistical summaries given the video frames as inputs. In order to alleviate the learning difficulty, we employ several spatial partitioning patterns to encode rough spatial locations instead of exact spatial Cartesian coordinates. Our approach is inspired by the observation that human visual system is sensitive to rapidly changing contents in the visual field, and only needs impressions about rough spatial locations to understand the visual contents. To validate the effectiveness of the proposed approach, we conduct extensive experiments with four 3D backbone networks, i.e., C3D, 3D-ResNet, R(2+1)D and S3D-G. The results show that our approach outperforms the existing approaches across these backbone networks on four downstream video analysis tasks including action recognition, video retrieval, dynamic scene recognition, and action similarity labeling. The source code is publicly available at: https://github.com/laura-wang/video_repres_sts. [ABSTRACT FROM AUTHOR]

Unsupervised landmark learning is the task of learning semantic keypoint-like representations without the use of expensive input keypoint annotations. A popular approach is to factorize an image into a pose and appearance data stream, then to reconstruct the image from the factorized components. The pose representation should capture a set of consistent and tightly localized landmarks in order to facilitate reconstruction of the input image. Ultimately, we wish for our learned landmarks to focus on the foreground object of interest. However, the reconstruction task of the entire image forces the model to allocate landmarks to model the background. Using a motion-based foreground assumption, this work explores the effects of factorizing the reconstruction task into separate foreground and background reconstructions in an unsupervised way, allowing the model to condition only the foreground reconstruction on the unsupervised landmarks. Our experiments demonstrate that the proposed factorization results in landmarks that are focused on the foreground object of interest when measured against ground-truth foreground masks. Furthermore, the rendered background quality is also improved as ill-suited landmarks are no longer forced to model this content. We demonstrate this improvement via improved image fidelity in a video-prediction task. Code is available at https://github.com/NVIDIA/UnsupervisedLandmarkLearning. [ABSTRACT FROM AUTHOR]

Camouflaged object detection (COD) aims to detect/segment camouflaged objects embedded in the environment, which has attracted increasing attention over the past decades. Although several COD methods have been developed, they still suffer from unsatisfactory performance due to the intrinsic similarities between the foreground objects and background surroundings. In this paper, we propose a novel Feature Aggregation and Propagation Network (FAP-Net) for camouflaged object detection. Specifically, we propose a Boundary Guidance Module (BGM) to explicitly model the boundary characteristic, which can provide boundary-enhanced features to boost the COD performance. To capture the scale variations of the camouflaged objects, we propose a Multi-scale Feature Aggregation Module (MFAM) to characterize the multi-scale information from each layer and obtain the aggregated feature representations. Furthermore, we propose a Cross-level Fusion and Propagation Module (CFPM). In the CFPM, the feature fusion part can effectively integrate the features from adjacent layers to exploit the cross-level correlations, and the feature propagation part can transmit valuable context information from the encoder to the decoder network via a gate unit. Finally, we formulate a unified and end-to-end trainable framework where cross-level features can be effectively fused and propagated for capturing rich context information. Extensive experiments on three benchmark camouflaged datasets demonstrate that our FAP-Net outperforms other state-of-the-art COD models. Moreover, our model can be extended to the polyp segmentation task, and the comparison results further validate the effectiveness of the proposed model in segmenting polyps. The source code and results will be released at https://github.com/taozh2017/FAPNet. [ABSTRACT FROM AUTHOR]

Few-shot segmentation aims at learning to segment query images guided by only a few annotated images from the support set. Previous methods rely on mining the feature embedding similarity across the query and the support images to achieve successful segmentation. However, these models tend to perform badly in cases where the query instances have a large variance from the support ones. To enhance model robustness against such intra-class variance, we propose a Double Recalibration Network (DRNet) with two recalibration modules, i.e., the Self-adapted Recalibration (SR) module and the Cross-attended Recalibration (CR) module. In particular, beyond learning robust feature embedding for pixel-wise comparison between support and query as in conventional methods, the DRNet further exploits semantic-aware knowledge embedded in the query image to help segment itself, which we call ‘self-adapted recalibration’. More specifically, DRNet first employs guidance from the support set to roughly predict an incomplete but correct initial object region for the query image, and then reversely uses the feature embedding extracted from the incomplete object region to segment the query image. Also, we devise a CR module to refine the feature representation of the query image by propagating the underlying knowledge embedded in the support image’s foreground to the query. Instead of foreground global pooling, we refine the response at each pixel in the query feature map by attending to all foreground pixels in the support feature map and taking the weighted average by their similarity; meanwhile, feature maps of the query image are also added back to weighted feature maps as a residual connection. Our DRNet can effectively address the intra-class variance under the few-shot setting with such two recalibration modules, and mine more accurate target regions for query images. We conduct extensive experiments on the popular benchmarks PASCAL- $5^{i}$ and COCO- $20^{i}$. The DRNet with the best configuration achieves the mIoU of $\textbf {63.6}\%$ and $\textbf {64.9}\%$ on PASCAL- $5^{i}$ and $\textbf {44.7}\%$ and $\textbf {49.6}\%$ on COCO- $20^{i}$ for 1-shot and 5-shot settings respectively, significantly outperforming the state-of-the-arts without any bells and whistles. Code is available at: https://github.com/fangzy97/drnet. [ABSTRACT FROM AUTHOR]

We study the colorization problem in monochrome-color dual-lens camera systems, i.e. colorizing the gray image from the monochrome camera using the color image from the color camera as reference. In related methods, cost volume based CNN methods achieve the state-of-the-art results, but they are costly in GPU memory due to building the 4D cost volume. Recently, some slice-wise cross-attention based methods are proposed for related problems. The slice-wise cross-attention has much less costs in GPU memory but directly using them for this colorization problem cannot generate competing results. We make use of the non-local computation property of cross-attention to propose a transformer based method. To overcome the limitations of straight-forward slice-wise cross-attention, we propose the spatially consistent cross-attention (SCCA) block to encourage pixels of slices across different epipolar lines in the gray image to find spatially consistent correspondence with pixels of the reference color image. And, to further reduce the memory cost while keeping the colorization accuracy, we design a pyramid processing strategy to cascade a series of SCCA blocks with smaller slice size and perform the colorization from coarse to fine. To extract more powerful image features, we use several regional self-attention (RSA) blocks with U-style connections. Experimental results show that we outperform the state-of-the-art methods largely on the synthesized datasets of Cityscapes, Sintel, and SceneFlow, and the real monochrome-color dual-lens dataset. [ABSTRACT FROM AUTHOR]

Image dehazing aims to remove haze in images to improve their image quality. However, most image dehazing methods heavily depend on strict prior knowledge and paired training strategy, which would hinder generalization and performance when dealing with unseen scenes. In this paper, to address the above problem, we propose Bidirectional Normalizing Flow (BiN-Flow), which exploits no prior knowledge and constructs a neural network through weakly-paired training with better generalization for image dehazing. Specifically, BiN-Flow designs 1) Feature Frequency Decoupling (FFD) for mining the various texture details through multi-scale residual blocks and 2) Bidirectional Propagation Flow (BPF) for exploiting the one-to-many relationships between hazy and haze-free images using a sequence of invertible Flow. In addition, BiN-Flow constructs a reference mechanism (RM) that uses a small number of paired hazy and haze-free images and a large number of haze-free reference images for weakly-paired training. Essentially, the mutual relationships between hazy and haze-free images could be effectively learned to further improve the generalization and performance for image dehazing. We conduct extensive experiments on five commonly-used datasets to validate the BiN-Flow. The experimental results that BiN-Flow outperforms all state-of-the-art competitors demonstrate the capability and generalization of our BiN-Flow. Besides, our BiN-Flow could produce diverse dehazing images for the same image by considering restoration diversity. [ABSTRACT FROM AUTHOR]

Relative colour constancy is an essential requirement for many scientific imaging applications. However, most digital cameras differ in their image formations and native sensor output is usually inaccessible, e.g., in smartphone camera applications. This makes it hard to achieve consistent colour assessment across a range of devices, and that undermines the performance of computer vision algorithms. To resolve this issue, we propose a colour alignment model that considers the camera image formation as a black-box and formulates colour alignment as a three-step process: camera response calibration, response linearisation, and colour matching. The proposed model works with non-standard colour references, i.e., colour patches without knowing the true colour values, by utilising a novel balance-of-linear-distances feature. It is equivalent to determining the camera parameters through an unsupervised process. It also works with a minimum number of corresponding colour patches across the images to be colour aligned to deliver the applicable processing. Three challenging image datasets collected by multiple cameras under various illumination and exposure conditions, including one that imitates uncommon scenes such as scientific imaging, were used to evaluate the model. Performance benchmarks demonstrated that our model achieved superior performance compared to other popular and state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Model-based single image dehazing algorithms restore haze-free images with sharp edges and rich details for real-world hazy images at the expense of low PSNR and SSIM values for synthetic hazy images. Data-driven ones restore haze-free images with high PSNR and SSIM values for synthetic hazy images but with low contrast, and even some remaining haze for real-world hazy images. In this paper, a novel single image dehazing algorithm is introduced by combining model-based and data-driven approaches. Both transmission map and atmospheric light are first estimated by the model-based methods, and then refined by dual-scale generative adversarial networks (GANs) based approaches. The resultant algorithm forms a neural augmentation which converges very fast while the corresponding data-driven approach might not converge. Haze-free images are restored by using the estimated transmission map and atmospheric light as well as the Koschmieder’s law. Experimental results indicate that the proposed algorithm can remove haze well from real-world and synthetic hazy images. [ABSTRACT FROM AUTHOR]

Vision-based fire detection systems have been significantly improved by deep models; however, higher numbers of false alarms and a slow inference speed still hinder their practical applicability in real-world scenarios. For a balanced trade-off between computational cost and accuracy, we introduce dual fire attention network (DFAN) to achieve effective yet efficient fire detection. The first attention mechanism highlights the most important channels from the features of an existing backbone model, yielding significantly emphasized feature maps. Then, a modified spatial attention mechanism is employed to capture spatial details and enhance the discrimination potential of fire and non-fire objects. We further optimize the DFAN for real-world applications by discarding a significant number of extra parameters using a meta-heuristic approach, which yields around 50% higher FPS values. Finally, we contribute a medium-scale challenging fire classification dataset by considering extremely diverse, highly similar fire/non-fire images and imbalanced classes, among many other complexities. The proposed dataset advances the traditional fire detection datasets by considering multiple classes to answer the following question: what is on fire? We perform experiments on four widely used fire detection datasets, and the DFAN provides the best results compared to 21 state-of-the-art methods. Consequently, our research provides a baseline for fire detection over edge devices with higher accuracy and better FPS values, and the proposed dataset extension provides indoor fire classes and a greater number of outdoor fire classes; these contributions can be used in significant future research. Our codes and dataset will be publicly available at https://github.com/tanveer-hussain/DFAN. [ABSTRACT FROM AUTHOR]

Most existing salient object detection (SOD) methods are designed for RGB images and do not take advantage of the abundant information provided by light fields. Hence, they may fail to detect salient objects of complex structures and delineate their boundaries. Although some methods have explored multi-view information of light field images for saliency detection, they require tedious pixel-level manual annotations of ground truths. In this paper, we propose a novel weakly-supervised learning framework for salient object detection on light field images based on bounding box annotations. Our method has two major novelties. First, given an input light field image and a bounding-box annotation indicating the salient object, we propose a ground truth label hallucination method to generate a pixel-level pseudo saliency map, to avoid heavy cost of pixel-level annotations. This method generates high quality pseudo ground truth saliency maps to help supervise the training, by exploiting information obtained from the light field (including depths and RGB images). Second, to exploit the multi-view nature of the light field data in learning, we propose a fusion attention module to calibrate the spatial and channel-wise light field representations. It learns to focus on informative features and suppress redundant information from the multi-view inputs. Based on these two novelties, we are able to train a new salient object detector with two branches in a weakly-supervised manner. While the RGB branch focuses on modeling the color contrast in the all-in-focus image for locating the salient objects, the Focal branch exploits the depth and the background spatial redundancy of focal slices for eliminating background distractions. Extensive experiments show that our method outperforms existing weakly-supervised methods and most fully supervised methods. [ABSTRACT FROM AUTHOR]

Codecs using spectral-spatial transforms efficiently compress raw camera images captured with a color filter array (CFA-sampled raw images) by changing their RGB color space into a decorrelated color space. This study describes two types of spectral-spatial transform, called extended Star-Tetrix transforms (XSTTs), and their edge-aware versions, called edge-aware XSTTs (EXSTTs), with no extra bits (side information) and little extra complexity. They are obtained by (i) extending the Star-Tetrix transform (STT), which is one of the latest spectral-spatial transforms, to a new version of our previously proposed wavelet-based spectral-spatial transform and a simpler version; (ii) considering that each 2D predict step of the wavelet transform is a combination of two 1D diagonal or horizontal-vertical transforms; (iii) weighting the transforms along the edge directions in the images. Compared with XSTTs, the EXSTTs can decorrelate CFA-sampled raw images well: they reduce the difference in energy between the two green components by about 3.38–30.08 % for high-quality camera images and 8.97–14.47 % for mobile phone images. The experiments on JPEG 2000-based lossless and lossy compression of CFA-sampled raw images show better performance than conventional methods. For high-quality camera images, the XSTTs/EXSTTs produce results equal to or better than the conventional methods: especially for images with many edges, the type-I EXSTT improves them by about 0.03–0.19 bpp in average lossless bitrate and the XSTTs improve them by about 0.16–0.96 dB in average Bjøntegaard delta peak signal-to-noise ratio. For mobile phone images, our previous work perform the best, whereas the XSTTs/EXSTTs show similar trends to the case of high-quality camera images. [ABSTRACT FROM AUTHOR]

The major challenge in high dynamic range (HDR) imaging for dynamic scenes is suppressing ghosting artifacts caused by large object motions or poor exposures. Whereas recent deep learning-based approaches have shown significant synthesis performance, interpretation and analysis of their behaviors are difficult and their performance is affected by the diversity of training data. In contrast, traditional model-based approaches yield inferior synthesis performance to learning-based algorithms despite their theoretical thoroughness. In this paper, we propose an algorithm unrolling approach to ghost-free HDR image synthesis algorithm that unrolls an iterative low-rank tensor completion algorithm into deep neural networks to take advantage of the merits of both learning- and model-based approaches while overcoming their weaknesses. First, we formulate ghost-free HDR image synthesis as a low-rank tensor completion problem by assuming the low-rank structure of the tensor constructed from low dynamic range (LDR) images and linear dependency among LDR images. We also define two regularization functions to compensate for modeling inaccuracy by extracting hidden model information. Then, we solve the problem efficiently using an iterative optimization algorithm by reformulating it into a series of subproblems. Finally, we unroll the iterative algorithm into a series of blocks corresponding to each iteration, in which the optimization variables are updated by rigorous closed-form solutions and the regularizers are updated by learned deep neural networks. Experimental results on different datasets show that the proposed algorithm provides better HDR image synthesis performance with superior robustness compared with state-of-the-art algorithms, while using significantly fewer training samples. [ABSTRACT FROM AUTHOR]

Underwater image enhancement aims at improving the visibility and eliminating color distortions of underwater images degraded by light absorption and scattering in water. Recently, retinex variational models show remarkable capacity of enhancing images by estimating reflectance and illumination in a retinex decomposition course. However, ambiguous details and unnatural color still challenge the performance of retinex variational models on underwater image enhancement. To overcome these limitations, we propose a hyper-laplacian reflectance priors inspired retinex variational model to enhance underwater images. Specifically, the hyper-laplacian reflectance priors are established with the $l_{1/2}$ -norm penalty on first-order and second-order gradients of the reflectance. Such priors exploit sparsity-promoting and complete-comprehensive reflectance that is used to enhance both salient structures and fine-scale details and recover the naturalness of authentic colors. Besides, the $l_{2}$ norm is found to be suitable for accurately estimating the illumination. As a result, we turn a complex underwater image enhancement issue into simple subproblems that separately and simultaneously estimate the reflection and the illumination that are harnessed to enhance underwater images in a retinex variational model. We mathematically analyze and solve the optimal solution of each subproblem. In the optimization course, we develop an alternating minimization algorithm that is efficient on element-wise operations and independent of additional prior knowledge of underwater conditions. Extensive experiments demonstrate the superiority of the proposed method in both subjective results and objective assessments over existing methods. The code is available at: https://github.com/zhuangpeixian/HLRP. [ABSTRACT FROM AUTHOR]

The aesthetic quality of an image is defined as the measure or appreciation of the beauty of an image. Aesthetics is inherently a subjective property but there are certain factors that influence it such as, the semantic content of the image, the attributes describing the artistic aspect, the photographic setup used for the shot, etc. In this paper we propose a method for the automatic prediction of the aesthetics of an image that is based on the analysis of the semantic content, the artistic style and the composition of the image. The proposed network includes: a pre-trained network for semantic features extraction (the Backbone); a Multi Layer Perceptron (MLP) network that relies on the Backbone features for the prediction of image attributes (the AttributeNet); a self-adaptive Hypernetwork that exploits the attributes prior encoded into the embedding generated by the AttributeNet to predict the parameters of the target network dedicated to aesthetic estimation (the AestheticNet). Given an image, the proposed multi-network is able to predict: style and composition attributes, and aesthetic score distribution. Results on three benchmark datasets demonstrate the effectiveness of the proposed method, while the ablation study gives a better understanding of the proposed network. [ABSTRACT FROM AUTHOR]