Search

Remote sensing image fusion plays important roles in numerous applications, including monitoring, metrology, and agriculture. Image fusion gathers essential information from several image sources and consolidates them into a single image called a fused image. The fused image involves relevant data, and it is more informative than any other images extracted from one source. This study proposed a pansharpening technique based on image filtering utilising a bilateral filter to generate high‐frequency details from panchromatic image. The various types of side window guided filters are employed to enhance the multispectral band from panchromatic image and then used these filters to adjust spatial data misfortune that happens when images are combined. Experimental results demonstrated that the proposed method provides consistent results concise with reported by the previous research in terms of subjective and objective assessments on remote sensing data.

Recently, deep learning based multispectral (MS) and panchromatic (PAN) image fusion methods have been proposed, which extracted features automatically and hierarchically by a series of non-linear transformations to model the complicated imaging discrepancy. But they always pay more attention to the extraction and compensation of spatial details and use the mean squared error or mean absolute error as a loss function, regardless of the preservation of spectral information contained in multispectral images. For the sake of the improvements in both spatial and spectral resolution, this paper presents a novel fusion model that takes the spectral preservation into consideration, and learns the spectral cues from the process of generating a spectrally refined multispectral image, which is constrained by a spectral loss between the generated image and the reference image. Then these spectral cues are used to modulate the PAN features to obtain final fusion result. Experimental results on reduced-resolution and full-resolution datasets demonstrate that the proposed method can obtain a better fusion result in terms of visual inspection and evaluation indices when compared with current state-of-the-art methods.

Contrastive self-supervised learning (CSSL) has achieved promising results in extracting visual features from unlabeled data. Most of the current CSSL methods are used to learn global image features with low-resolution that are not suitable or efficient for pixel-level tasks. In this paper, we propose a coarse-to-fine CSSL framework based on a novel contrasting strategy to address this problem. It consists of two stages, one for encoder pre-training to learn global features and the other for decoder pre-training to derive local features. Firstly, the novel contrasting strategy takes advantage of the spatial structure and semantic meaning of different regions and provides more cues to learn than that relying only on data augmentation. Specifically, a positive pair is built from two nearby patches sampled along the direction of the texture if they fall into the same cluster. A negative pair is generated from different clusters. When the novel contrasting strategy is applied to the coarse-to-fine CSSL framework, global and local features are learned successively by forcing the positive pair close to each other and the negative pair apart in an embedding space. Secondly, a discriminant constraint is incorporated into the per-pixel classification model to maximize the inter-class distance. It makes the classification model more competent at distinguishing between different categories that have similar appearance. Finally, the proposed method is validated on four SAR images for land-cover classification with limited labeled data and substantially improves the experimental results. The effectiveness of the proposed method is demonstrated in pixel-level tasks after comparison with the state-of-the-art methods.

Recent studies have shown that rutin (Quercetin-3-O-rhamnosylglucoside) may have an inhibitory effect on COVID-19. Rutin can be extracted from Tartary buckwheat as an active pharmaceutical ingredient. Nevertheless, its purification is mainly hindered by Kaempferol-3-O-rutinoside (K3R) due to their similar molecular structures. This study intends to propose a simulated moving bed (SMB) chromatography process of rutin and K3R to achieve continuous production. True moving bed (TMB) and SMB models were established to numerically analyze and optimize this process. The system consists of a four-zone SMB with two columns in each zone. The effects of the switch interval, feed flowrate, desorbent flowrate, extract flowrate, raffinate flowrate, and recycle flowrate on the purity and yield of rutin and K3R were investigated and the optimized conditions were chosen as 5 min, 3.5, 40, 34, 9.5, and 24.5 L/min, respectively. Consequently, the purities of 99.64% and 99.25%, and the yields of 99.81% and 99.37% for rutin and K3R were obtained, respectively. The simulation results can provide a guidance for the future industrial application of SF-SMB to separate rutin and K3R.

Image compression usually includes two important operations: compression and decompression. The compression process includes the operation of discarding information, while the decompression process is to retrieve the lost information. In order to make the decompressed image more similar to the original image, the classic compression methods generally adopt the process of approximately reversible compression and decompression. Inspired by the symmetric structure in classic compression methods, we propose a new symmetrical lattice generating adversarial network (SLGAN) for the remote sensing images (RSIs) compression in this paper. Several pairs of symmetrical encoder-decoder lattices are constructed to build a generator to first generate deep representative codes of images and then decode them. For each pair of encoded lattice and decoded lattice, one discriminator is constructed to perform adversarial learning with the generator. When multiple discriminators are used for all the lattices, a cooperative learning algorithm is proposed to train jointly pairs of symmetric lattices in the generator. Moreover, to enhance edges, contours, and textures in the decomposed RSIs, an enhanced Laplacian of gaussian (ELoG) loss is designed as a regularizer to train the SLGAN. Experimental results on the panchromatic images from GF2 satellite show that SLGAN outperforms other existing state-of-the-art methods.

With the increasing accessibility of remote sensing videos, remote sensing tracking is gradually becoming a hot issue. However, accurately detecting and tracking in complex remote sensing scenes is still a challenge. In this paper, we propose a collaborative learning tracking network for remote sensing videos, including a consistent receptive field parallel fusion module (CRFPF), dual-branch spatial-channel co-attention (DSCA) module, and geometric constraint re-track strategy (GCRT). Considering the small-size objects of remote sensing scenes are difficult for general forward networks to extract effective features, we propose a CRFPF-module to establish parallel branches with consistent receptive fields to separately extract from shallow to deep features and then fuse hierarchical features adaptively. Since the objects and their background are difficult to distinguish, the proposed DSCA-module uses the spatial-channel co-attention mechanism to collaboratively learn the relevant information, which enhances the saliency of the objects and regresses to precise bounding boxes. Considering the interference of similar objects, we designed a GCRT-strategy to judge whether there is a false detection through the estimated motion trajectory and then recover the correct object by weakening the feature response of interference. The experimental results and theoretical analysis on multiple data sets demonstrate our proposed method's feasibility and effectiveness. Code and net are available at https://github.com/Dawn5786/CoCRF-TrackNet.

Supervised change detection methods always face a big challenge that the current scene (target domain) is fully unlabeled. In remote sensing, it is common that we have sufficient labels in another scene (source domain) with a different but related data distribution. In this article, we try to detect changes in the target domain with the help of the prior knowledge learned from multiple source domains. To achieve this goal, we propose a change detection framework based on selective adversarial adaptation. The adaptation between multisource and target domains is fulfilled by two domain discriminators. First, the first domain discriminator regards each scene as an individual domain and is designed for identifying the domain to which each input sample belongs. According to the output of the first domain discriminator, a subset of important samples is selected from multisource domains to train a deep neural network (DNN)-based change detection model. As a result, not only the positive transfer is enhanced but also the negative transfer is alleviated. Second, as for the second domain discriminator, all the selected samples are thought from one domain. Adversarial learning is introduced to align the distributions of the selected source samples and the target ones. Consequently, it further adapts the knowledge of change from the source domain to the target one. At the fine-tuning stage, target samples with reliable labels and the selected source ones are used to jointly fine-tune the change detection model. As the target domain is fully unlabeled, homogeneity- and boundary-based strategies are exploited to make the pseudolabels from a preclassification map reliable. The proposed method is evaluated on three SAR and two optical data sets, and the experimental results have demonstrated its effectiveness and superiority.

Depth image super-resolution (DISR) is a significant yet challenging task. In this paper, we propose a novel edge-guided framework for color-guided DISR aiming at reducing the artifacts caused by the introduced color image. Considering the different view synthesis characteristics of texture and smooth regions in depth image, we propose that the edge and smooth regions of depth map should be reconstructed in different ways. In our framework, a novel joint trilateral filter is built firstly, which has two different modes: one for the pixels on the edges and the other for the pixels in the smooth regions. Secondly, in each filtering iteration during the whole upsampling process, we use the edge map updated by the upsampled depth map as a guidance to decide when to change the filter mode. Benefiting from the strategy, the high-resolution depth map reconstructed has less texture copying and contains sharp and smooth edges. Experimental results demonstrate the effectiveness of our approach over prior depth map upsampling works.

Researchers have carried out many approaches to recognizing the OCR through software-based and FPGA-based and its relationship with the ANNs and training these NNs by using the BP algorithm. The FPGA supports high speed to recognize the character because it works in parallel. It involves many logic circuits and can process at the same time (clock cycle). There have been problems with the established system because the size, shape, and style of supposedly identical characters might differ from person to person and even within the same person on rare occasions. The photograph is vulnerable to noise and can lose some or all of its visual clarity as a result. There are strict guidelines that govern the way a character looks that the character must adhere to. The NN and its learning process could only be defined by heuristically deducing its rules from samples and the enormous dataset.

Feature representation has received more and more attention in image classification. Existing methods always directly extract features via convolutional neural networks (CNNs). Recent studies have shown the potential of CNNs when dealing with images' edges and textures, and some methods have been explored to further improve the representation process of CNNs. In this article, we propose a novel classification framework called the multiscale curvelet scattering network (MSCCN). Using the multiscale curvelet-scattering module (CCM), image features can be effectively represented. There are two parts in MSCCN, which are the multiresolution scattering process and the multiscale curvelet module. According to multiscale geometric analysis, curvelet features are utilized to improve the scattering process with more effective multiscale directional information. Specifically, the scattering process and curvelet features are effectively formulated into a unified optimization structure, with features from different scale levels being efficiently aggregated and learned. Furthermore, a one-level CCM, which can essentially improve the quality of feature representation, is constructed to be embedded into other existing networks. Extensive experimental results illustrate that MSCCN achieves better classification accuracy when compared with state-of-the-art techniques. Eventually, the convergence, insight, and adaptability are evaluated by calculating the trend of loss function's values, visualizing some feature maps, and performing generalization analysis.

In the last five years, deep learning has been introduced to tackle the hyperspectral image (HSI) classification and demonstrated good performance. In particular, the convolutional neural network (CNN)-based methods for HSI classification have made great progress. However, due to the high dimensionality of HSI and equal treatment of all bands, the performance of these methods is hampered by learning features from useless bands for classification. Moreover, for patchwise-based CNN models, equal treatment of spatial information from the pixel-centered neighborhood also hinders the performance of these methods. In this article, we propose an end-to-end residual spectral–spatial attention network (RSSAN) for HSI classification. The RSSAN takes raw 3-D cubes as input data without additional feature engineering. First, a spectral attention module is designed for spectral band selection from raw input data by emphasizing useful bands for classification and suppressing useless bands. Then, a spatial attention module is designed for the adaptive selection of spatial information by emphasizing pixels from the same class as the center pixel or those are useful for classification in the pixel-centered neighborhood and suppressing those from a different class or useless. Second, two attention modules are also used in the following CNN for adaptive feature refinement in spectral–spatial feature learning. Third, a sequential spectral–spatial attention module is embedded into a residual block to avoid overfitting and accelerate the training of the proposed model. Experimental studies demonstrate that the RSSAN achieved superior classification accuracy compared with the state of the art on three HSI data sets: Indian Pines (IN), University of Pavia (UP), and Kennedy Space Center (KSC).

Most of the available graph-based semisupervised hyperspectral image classification methods adopt the cluster assumption to construct a Laplacian regularizer. However, they sometimes fail due to the existence of mixed pixels whose recorded spectra are a combination of several materials. In this paper, we propose a geometric low-rank Laplacian regularized semisupervised classifier, by exploring both the global spectral geometric structure and local spatial geometric structure of hyperspectral data. A new geometric regularized Laplacian low-rank representation (GLapLRR)-based graph is developed to evaluate spectral-spatial affinity of mixed pixels. By revealing the global low-rank and local spatial structure of images via GLapLRR, the constructed graph has the characteristics of spatial–spectral geometry description, robustness, and low sparsity, from which a more accurate classification of mixed pixels can be achieved. The proposed method is experimentally evaluated on three real hyperspectral datasets, and the results show that the proposed method outperforms its counterparts, when only a small number of labeled instances are available.

Artificial visual memories (AVMs) are crucial to many technologies, including artificial intelligence, humanoid robots, and visual prostheses. Most studies used a thick polycrystalline film as the AVM active layer to achieve a continuous channel for charge collection. However, because of the inherent interlayer shielding effect in thick films, both photogating and electrical gating are inefficient and the photoresponse of these AVMs is extremely low. Herein, the obstacle of the interlayer shielding effect in thick films is overcome by using molecularly thin two-dimensional molecular crystals (2DMCs). Because of the highly efficient photogating and electrical gating effects of 2D channels, the developed AVMs based on few-layered 2DMCs show record-high photoresponse. The AVMs also exhibited multilevel non-volatile memory functionality, with long retention time and excellent write–erase stability, by exploiting the persistent photoconductivity effect.

The recent years has witnessed a rapid development of Deep Learning (DL) based Automation Modulation Classification (AMC) methods, which has proved to outperform traditional classification approaches. In order to disturb the deep neural networks for AMC, in this paper, we propose an adversarial attack method to generate fake signals for fooling DL-based classifiers. Firstly, some constraints on visual difference and recoverability of fake signals are defined. Next, a Few Data Point Attacker (FDPA) is proposed to generate fake signals with few perturbed data points via differential evolution algorithm. Some experiments are taken on a public dataset, RML 2016.10a, and the results show that fake signals generated by the FDPA can remarkably reduce the accuracies of three types of DL-based AMC classifiers, a Convolutional Neural Network (CNN) based classifier, a Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) based classifier, and a classifier combined with CNN and LSTM-RNN. The code will be available.

Mycotoxin contamination causes disease and death in both humans and animals. Monascus Red, produced by Monascus purpureus, is used as a food colorant. However, its application is limited by contami...

In recent years, with the diversification of acquisition methods of very high resolution panchromatic (PAN) and multispectral (MS) remote sensing images, multiresolution remote sensing classification has become a research hotspot. In this paper, from the perspective of data–driven deep learning, we design a dual–branch attention fusion deep network (DBAF–Net) for the multiresolution classification. It aims to integrate the feature–level fusion and classification into an end–to–end network model. In the process of establishing a training sample library, unlike the traditional pixel–centric sampling strategy with fixed patch size, we propose an adaptive center–offset sampling strategy (ACO–SS), which allows each patch to adaptively determine the neighborhood range by finding the texture structure of the pixel to be classified. And the neighborhood range is not symmetrical with this pixel, we expect to capture the neighborhood information that is more conducive to its classification. In network structure, based on the captured patches by ACO–SS, we design a spatial attention module (SA–module) for PAN data and a channel attention module (CA–module) for MS data, thus highlighting the spatial resolution advantages of PAN data and the multi–channel advantages of MS data, respectively. Then these two features are interfused to improve and strengthen the fusion features in both spatial and channel. The quantitative and qualitative experimental results verify the robustness and effectiveness of the proposed method.

Doping is a critically important strategy to modulate the properties of organic semiconductors (OSCs) to improve their optoelectrical performances. Conventional bulk doping involves the incorporation of foreign molecular species ( i.e. , dopants) into the lattice of the host OSCs, and thus disrupts the packing of the host OSCs and induces structural defects, which tends to reduce the mobility and (or) the on/off ratio in organic field-effect transistors (OFETs). In this article, we report a highly efficient and highly controllable surface doping strategy utilizing 2D molecular crystals (2DMCs) as dopants to boost the mobility and to modulate the threshold voltage of OFETs. The amount of dopants, i.e. , the thickness of the 2DMCs, is controlled at monolayer precision, enabling fine tuning of the electrical properties of the OSCs at unprecedented accuracy. As a result, a prominent increase of the average mobility from 1.31 to 4.71 cm2 V−1 s−1 and a substantial reduction of the threshold voltage from −18.5 to −1.8 V are observed. Meanwhile, high on/off ratios of up to 108 are retained.

In this paper, a co-learning saliency detection method is proposed via coupled channels and low-rank factorization, by imitating the structural sparse coding and cooperative processing mechanism of two dorsal “where” and ventral “what” pathways in human vision system (HVS). First, images are partitioned into some superpixels and their structural sparsity are explored to locate pure background from image borders. Second, images are processed by two cortical pathways, to cooperatively learn a “where” feature map and a “what” feature map, by taking the background as the dictionary and using sparse coding errors as an indication of saliency. Finally, two feature maps are integrated to generate saliency map. Because the “where” and “what” feature maps are complementary to each other, our method can highlight the salient region and restrain the background. Some experiments are taken on several public benchmarks and the results show its superiority to its counterparts.

The extreme learning machine (ELM) has drawn increasing attention in the field of machine learning due to its high accuracy and efficient learning. However, classical ELM works in batch and passive learning paradigms, which cannot deal with sequential data effectively. ELM has been extended to online sequential learning form (OS-ELM) and active learning form (AL-ELM), in which the former is used to improve training efficiency and the latter is mainly adopted to improve accuracy. In order to solve the problem of labeling samples difficulty and costly, poor sample validity, and continuous iterative learning in polarimetric synthetic aperture radar (PolSAR) image classification, we propose an online active extreme learning machine (OA-ELM) algorithm to combine the strengths and make up the weaknesses of OS-ELM and AL-ELM, which improves both efficiency and generalization ability. OA-ELM can learn from sequential data dynamically with low computational complexity and good generalization ability. Specifically, OA-ELM reduces time and memory cost for training via extended recursive least squares for optimization. It also improves accuracy using informative training samples selected by proposed discrepancy sampling (DS), which modifies an active query method called margin sampling (MS). Before applying MS to ELM, real-valued outputs of ELM need to be converted into probabilistic outputs first. Instead, the proposed DS can be applied to ELM directly by calculating the difference between the two largest actual nonprobabilistic outputs of ELM. Experimental results of PolSAR classification demonstrate that OA-ELM is effective and efficient compared with other algorithms in terms of accuracy and running time.

A recent trend of automatic modulation classification is to automatically learn high-level abstraction of signals, instead of manually designing features for further classification. In this paper, we propose a new deep geometric convolutional network (DGCN) to hierarchically extract discriminative features from Wigner–Ville distribution map of signals. A group of geometric filters are constructed from a least square support vector machine, to capture the linear singularity existed in maps. The filters are cascaded to construct a deep network for extracting discriminative features and classifying signals with different modulation types. Some experiments are taken to investigate the performance of DGCN, and the results show that it can achieve high accuracy in classifying 15 types of modulation signals.

Desired by sparse subset learning, in this paper, a hyperspectral band selection method via pairwise band agreement with spatial-spectral graph regularier, referred as Regularized Band Selection via Learned Pairwise Agreement (RBS-LPA), was proposed. The process was formulated as a graph-regularized row-sparse constrained optimization problem, which select a few representative bands to code the all bands based on the learned pairwise band agreement. In RBS-LPA, a spatial-spectral informative graph, constructed by spatial-spectral neighbor relationship, is incorporated to encode both the spatial and spectral geometrical structure. By combining the learning procedure with graph regularizer jointly, the graph regularizer can adaptively change with the sparse representative bands, which ensures that the selected bands can well preserve the local manifold structure. Experimental results on three real urban hyperspectral data demonstrate the efficiency of the proposed RBS-LPA and achieve convincing performance.

Compared with natural image super-resolution, hyperspectral image super-resolution (HSR) is more complex because the redundancy in spectral bands and spatial information. To overcome the difficulties exist in HSR, in this paper, we propose a tensor spatial-spectral joint correlation based HSR method. Start with the tensor representation, we construct a series of fourth-order tensors to preserve the intrinsic structure of hyperspectral images, and then explore the spatial-spectral joint correlation based on meaningful interpretations of tensor canonical matrices. To further constrain the spectral characteristics, we analyze the sparsity of the spectral gradients and model it with Laplacian prior. Then, the two regularizations are combined with the reconstruction model to develop a new HSR method. Finally, an iterative optimization algorithm based on alternating direction method of multiplier (ADMM) and augmented Lagrangian multiplier method is proposed to reconstruct the high-resolution hyperspectral images. Experimental results on several data sets illustrate the effectiveness of our proposed method both in visual and numerical comparisons.

Remote Sensing Images (RSIs) often have extremely wide width and abundant terrain. In order to achieve rapid object detection in large RSIs, in this paper, a Deep Hash Assisted Network (DHAN) is constructed by introducing a hashing encoding of images in a two-stage deep neural network. Different with the available detection networks, DHAN first locates candidate object regions and then transfers the learned features to another Region Proposal Network (RPN) for detection. On the one hand, it can avoid the calculations on the background irrelevant to objects. On the other hand, the built hash encoding layer in DHAN can accelerate the detection via binary hash features. Moreover, a self attention layer is designed and combined with the convolution layer, to distinguish relatively small objects regions from a very large scene. The proposed method is tested on several public data sets, and the comparison results show that DHAN can remarkably improve the detection efficiency on large RSIs and simultaneously achieve high detection accuracy.

In this paper, a new Separable Attention Capsule Network (SACN) is proposed for signal classification. SACN is a light-weight network composed of multi-channel separable convolution layer, attention module and classification layer. First, depth-wise convolution is employed to extract features of signals in a low-complexity manner, and the multi-channel network structure is designed to increase the network width to improve the diversity of features of signals. Then a channel attention module is followed by a capsule network whose element contains a group of neurons. This attention module can explore the interdependence among channels to use global information to selectively strengthen some important channels, thus achieving the improvement of generalization ability of SACN. Some experiments are taken on several datasets with communication and radar signals, and the comparison results prove the efficiency of SACN and the superiority to its counterparts.

Tensor representation is the most natural and effective way to preserve the structural information of hyperspectral image (HSI), and thus is very beneficial to HSI processing. This paper represents the spectral features of each testing pixel and its spatial neighbors as a Spatial Neighbor Tensor (SNT), whose spectral vectors can be simultaneously sparse coded by the spectral vectors of a few common training pixels. The obtained sparse coding coefficients could be regarded as a Slice Sparse Coding Tensor (SSCT), which can be adaptively learnt and utilized to predict the labels of HSI pixels. Furthermore, to improve the efficiency, the Compressive Dimensionality Reduction (CDR) is introduced into tensor slice sparse coding for optimizing SSCT and thus an SSCT based Classifier with CDR (SSCTC-CDR) is proposed for hyperspectral image classification (HIC). The performance of SSCTC-CDR is evaluated on three real HSI data, and the results show that it can obtain high-accurate classification result with relatively low computation lost.

Organic phototransistors (OPTs) with the ability to convert light signals into electrical signals have drawn attention for a variety of potential multifunctional applications, such as images, night vision, and optical communications, because of their advantages including designable molecular structures and low-cost and large-area fabrication. However, the performances of OPTs are hampered by organic semiconducting architectures with low crystallinity, random crystallographic orientation, and stochastic position arising from the uncontrollable dewetting dynamics, which is detrimental to the transport of carriers and separation of excitons. In this work, a capillary-bridge lithography strategy to fabricate 1D single-crystal arrays with the feature of high crystallinity, strict alignment, and accurate positioning is utilized. Owing to the application periodical micropillars of the template with asymmetric wettability, capillary bridges with unidirectional dewetting behavior are achieved to fabricate large-area single-crystalline arrays with precise alignment and pure crystallographic orientation. As a result, encouraged by the high hole mobility of 13.52 cm2 V−1 s−1, high-performance OPTs with photosensitivity up to 2.5 × 107, responsivity up to 8.8 × 104 A W−1, and specific detectivity up to 7.5 × 1015 Jones are constructed. This research provides a guide for patterning large-area 1D single-crystalline arrays toward various high-performance multifunctional optoelectronic devices. This work was supported by the National Natural Science Foundation (51922012 and 21633014), the MOST of China (Grant Nos. 2017YFA0204504 and 2018YFA0208502, 2018YFA0704803), and Youth Innovation Promotion Association CAS (2018034).