Search

Deep learning has been the state-of-the-art solution to numerous remote sensing tasks, especially for building extraction. However, the performance of learning-based building extraction approaches depend to a large extent on the similarity of the source and target domain data. To alleviate the dependence on annotated data, and to exploit the potential of multimodal remote sensing data, a 3-D assisted semisupervised method for building extraction is proposed. The proposed method is based on self-training, a semisupervised method that utilizes both labeled and unlabeled data. In addition, photogrammetric digital surface model and belief function are exploited to bridge the domain gaps between the source and target data. The performance is evaluated with ISPRS Potsdam and Vaihingen benchmark datasets, and a WorldView-2 satellite multimodal dataset. Compared with the direct cross-domain test baseline, improvement of Jaccard score ranging from 8.91% to 21.39% is achieved, demonstrating the efficacy of the proposed 3-D self-training method.

The anthropogenic climate crisis results in the gradual loss of tree species in locations where they were previously able to grow. This leads to increasing workloads and requirements for foresters and arborists as they are forced to restructure their forests and city parks. The advancements in computer vision (CV)—especially in supervised deep learning (DL)—can help cope with these new tasks. However, they rely on large, carefully annotated datasets to produce good and generalizable models. This paper presents BAMFORESTS: a dataset with 27,160 individually delineated tree crowns in 105 ha of very-high-resolution UAV imagery gathered with two different sensors from two drones. BAMFORESTS covers four areas of coniferous, mixed, and deciduous forests and city parks. The labels contain instance segmentations of individual trees, and the proposed splits are balanced by tree species and vitality. Furthermore, the dataset contains the corrected digital surface model (DSM), representing tree heights. BAMFORESTS is annotated in the COCO format and is especially suited for training deep neural networks (DNNs) to solve instance segmentation tasks. BAMFORESTS was created in the BaKIM project and is freely available under the CC BY 4.0 license.

Accurate estimation of above-ground biomass (AGB) in forested areas is essential for studying forest ecological functions, surface carbon cycling, and global carbon balance. Over the past decade, models that harness the distinct features of multi-source remote sensing observations for estimating AGB have gained significant popularity. It is worth exploring the differences in model performance by using simple and fused data. Additionally, quantitative estimation of the impact of high-cost laser point clouds on satellite imagery of varying costs remains largely unexplored. To address these challenges, model performance and cost must be considered comprehensively. We propose a comprehensive assessment based on three perspectives (i.e., performance, potential and limitations) for four typical AGB-estimation models. First, different variables are extracted from the multi-source and multi-resolution data. Subsequently, the performance of four regression methods is tested for AGB estimation with diverse indicator combinations. Experimental results prove that the combination of multi-source data provides a highly accurate AGB regression model. The proposed regression and variables rating approaches can flexibly integrate other data sources for modeling. Furthermore, the data cost is discussed against the AGB model performance. Our study demonstrates the potential of using low-cost satellite data to provide a rough AGB estimation for larger areas, which can allow different remote sensing data to meet different needs of forest management decisions.

Detection and vectorization of windows from building façades are important for building energy modeling, civil engineering, and architecture design. However, current applications still face the challenges of low accuracy and lack of automation. In this article we propose a new two-steps workflow for window segmentation and vectorization from façade images. First, we propose a cross field learning-based neural network architecture, which is augmented by a grid-based self-attention module for window segmentation from rectified façade images, resulting in pixel-wise window blobs. Second, we propose a regression neural network augmented by squeeze-and-excitation (SE) attention blocks for window vectorization. The network takes the segmentation results together with the original façade image as input, and directly outputs the position of window corners, resulting in vectorized window objects with improved accuracy. In order to validate the effectiveness of our method, experiments are carried out on four public façades image datasets, with results usually yielding a higher accuracy for the final window prediction in comparison to baseline methods on four datasets in terms of intersection over union score, F1 score, and pixel accuracy.

Summary: Resin embedding combined with ultra-thin sectioning has been widely used in microscopic and electron imaging to acquire precise structural information of biological tissues. However, the existing embedding method was detrimental to quenchable fluorescent signals of precise structures and pH-insensitive fluorescent dyes. Here, we developed a low-temperature chemical polymerization method named HM20-T to maintain weak signals of various precise structures and to decrease background fluorescence. The fluorescence preservation ratio of green fluorescent protein (GFP) tagged presynaptic elements and tdTomato labeled axons doubled. The HM20-T method was suitable for a variety of fluorescent dyes, such as DyLight 488 conjugated Lycopersicon esculentum lectin. Moreover, the brains also retained immunoreactivity after embedding. In summary, the HM20-T method was suitable for the characterization of multi-color labeled precise structures, which would contribute to the acquisition of complete morphology of various biological tissues and to the investigation of composition and circuit connection in the whole brain.

Although deep learning techniques have brought unprecedented accuracy to automatic building extraction, several main issues still constitute an obstacle to effective and practical applications. The industry is eager for higher accuracy and more flexible data usage. In this paper, we present a co-learning framework applicable to building extraction from optical images and photogrammetric point clouds, which can take the advantage of 2D/3D multimodality data. Instead of direct information fusion, our co-learning framework adaptively exploits knowledge from another modality during the training phase with a soft connection, via a predefined loss function. Compared to conventional data fusion, this method is more flexible, as it is not mandatory to provide multimodality data in the test phase. We propose two types of co-learning: a standard version and an enhanced version, depending on whether unlabeled training data are employed. Experimental results from two data sets show that the methods we present can enhance the performance of both image and point cloud networks in few-shot tasks, as well as image networks when applying fully labeled training data sets.

Low/high temperature stress is one of the key abiotic factors restricting the implementation of microalgal large-scale production. Metabolism, involved in abiotic stress resistances, have been reported in plants, but the related research in microalgae is far behind. This study analyzed the transcriptional changes in Nannochloropsis oceanica under low and high temperature stresses by RNA-seq technology. Results indicated that the total differentially expressed gene (DEG) amount under high temperature stress was much more than low temperature stress. GO enrichment analysis suggested that DEGs under low temperature were mainly concentrated in protein modification and phosphorus metabolic process, while transmembrane transport, lipid localization and transport, and cellular homeostasis were the most enriched terms under high temperature. Moreover, “membrane” was the most enriched term in both low and high temperature groups. The gene expressions of 37 stress resistance-related enzymes/proteins (172 genes) were analyzed concretely in N. oceanica for the first time. Results revealed that ornithine decarboxylase (ODC) was significantly down-regulated, while choline dehydrogenase (CHDH), late embryogenesis abundant (LEA), dehydrin (DHN), chitinase (CHI), calmodulin (CaM), and lipid transfer protein (LTP) were extremely up-regulated under low temperature. Under high temperature, more genes with drastic changes were identified. Superoxide dismutase (SOD), ascorbate peroxidase (APX), glutathione synthetase (GSS), and glyoxalase II (Gly II) exhibited down-regulated significantly, while glutathione S-transferase (GST), CHDH, L-asparaginase (L-ASP), LEA, DHN, early response to dehydration protein (ERD), heat shock protein (HSP), and CaM were dramatically up-regulated. No significant changes were detected in glutathione peroxidase (GPX), tocopherol cyclase (TC), gamma-glutamylcysteine synthetase (γ-GCS), pyrroline‐5–carboxylate reductase (P5CR), asparagine synthetase (ASN ), succinate-semialdehyde dehydrogenase (SSADH), pyruvate decarboxylase (PDC), and S-adenosylmethionine synthetase (SAMS) under low or high temperature. Furthermore, 166 transcription factor (TF) genes related to plant stress resistance were characterized and categorized into 20 TF families. MYB was the most abundant and active one, revealing that the MYB family played a critical role of temperature resistance in N. oceanica. These new findings could benefit the understanding of the temperature resistance mechanism in Nannochloropsis. The significantly changed genes might be ideal candidates for breeding elite Nannochloropsis strains with stronger resistance to temperature stress using genetic engineering methods.

Multispectral sensors are important instruments for Earth observation. In remote sensing applications, the near-infrared (NIR) band, together with the visible spectrum (RGB), provide abundant information about ground objects. However, the NIR band is typically not available on low-cost camera systems, which presents challenges for the vegetation extraction. To this end, this paper presents a conditional generative adversarial network (cGAN) method to simulate the NIR band from RGB bands of Sentinel-2 multispectral data. We adapt a robust loss function and a structural similarity index loss (SSIM) in addition to the GAN loss to improve the model performance. With 45,529 multi-seasonal test images across the globe, the simulated NIR band had a mean absolute error of 0.02378 and an SSIM of 89.98%. A rule-based landcover classification using the simulated normalized difference vegetation index (NDVI) achieved a Jaccard score of 89.50%. The evaluation metrics demonstrated the versatility of the learning-based paradigm in remote sensing applications. Our simulation approach is flexible and can be easily adapted to other spectral bands.

Interactions between aboveground and belowground ecosystems components modulate ecosystem functions. However, few studies have focused on the response of soil nematodes to plantation reconstruction. In this study, Cinnamomum longipaniculatum was planted in the gap of a Pinus massoniana plantation. Based on the size, center and edge of C. longipaniculatum patches, the characteristics of the soil nematode community were studied after 8 years of plantation reconstruction. The effect of the C. longipaniculatum plantation on the soil nematode community was related to patch size. The soil nematode community did not change significantly with increasing patch size above 30 m × 30 m. The soil recovered better in the patch center than at the edge, but the edge increased the diversity of soil nematodes with low C-P values. The experimental treatment increased the dominance of soil plant parasite nematodes and bacterivores, decreased the trophic diversity index, and increased the channel ratio, maturity index, enrichment index and structure index of soil nematodes, indicating that soil nutrient cycling improved and the diversity of the soil food web increased. After P. massoniana plantation gaps were replanted with C. longipaniculatum, the main soil properties that affected the soil nematode community were organic carbon, pH, water content and soil aggregate degree. In the soil nematode community, fungal nematodes had the strongest influence on omnivorous-predatory nematodes. Our study showed that the soil environment changed with vegetation and affected soil nematode groups, and soil nematodes effectively reflected soil ecological restoration after P. massoniana plantation gaps were replanted with C. longipaniculatum.

In this study, a microcapsule diet was produced by spray-drying. Then, the filtration rate (FR) and ingestion rate (IR) of manila clam Ruditapes philippinarum were determined in relation to feeding on the microcapsule and microalgal Chaetoceros muelleri diets. Finally, two feeding experiments were conducted to evaluate the effects of increasing substitution of C. muelleri by the microcapsule on the survival rates, growth performances, biochemical compositions, and specific activities of digestive enzymes of two clams, R. philippinarum and razor clam Sinonovacula constricta. In experiment 1, the replacement of C. muelleri with the microcapsule was evaluated at three substitution levels (0%, 50%, and 100%) with R. philippinarum. In experiment 2, the replacement was evaluated at five substitution levels (0%, 25%, 50%, 75%, and 100%) with S. constricta. Results showed that the FR and IR of R. philippinarum in the microcapsule suspension were (4.07 ± 0.19) mL clam−1 min−1 and (6.36 ± 0.19) × 104 microcapsules clam−1 min−1, respectively. In experiment 1 and 2, no significant difference in survival rate of R. philippinarum or S. constricta was observed between different groups. The partial replacement of C. muelleri (25–50%) by the microcapsule diet did not significantly reduce the growth rates of these clams, while higher replacement (50–100%) did. The high-level microcapsule diet led to higher carbohydrate content when compared with high-level C. muelleri diet. In experiment 1, no significant difference in specific activity of amylase or lipase of R. philippinarum was observed between the 50% and 0% microcapsule groups. Meanwhile, no significant difference in specific activity of protease or lipase of S. constricta was observed between the 50% and 0% microcapsule groups in experiment 2. Our result suggested that the microcapsule was effective as a partial replacement of C. muelleri at up to 25–50% of dry weight of C. muelleri in clams.

Accurate building extraction from remotely sensed images is essential for topographic mapping, urban planning, disaster management, navigation, and many other applications [...]

Individual tree crown (ITC) segmentation supports numerous applications in forest management and ecology. In the latter context, special attention is dedicated to the study of angular reflection effects, caused by the interaction of incident sunlight with a canopy. High precision airborne analysis of these effects requires multi-view sensor systems and multi-view ITC segmentation. In particular oblique view image segmentation is difficult and has been addressed by numerous template based methods. This contribution identifies persistent shortcomings in the state of the art and tackles the problem by a multi-step workflow utilizing the digital surface model (DSM), derived from multi-view stereo data. A slightly revised version of the previously published levelset-watershed segmentation of the DSM is presented as the first step. In the second step, the contour of the visible part of a candidate tree in images with known orientation is obtained by means of ray casting and concave hull calculation. The method was tested on a deciduous, mixed and coniferous plot, whose aerial images were acquired using the 3K camera system in 2018 at Kranzberg Forest, Bavaria, Germany. Accuracies were assessed at hand of human operator generated groundtruth tree tops for the DSM as well as images with zenit angles of approximately 0,45 and 52 degrees. The resulting F1-scores, averaged over the plots, are 0.909/0.902/0.886/0.876 for the DSM/near-nadir-/oblique-/maximum-oblique-images, respectively.

Dense matching plays a crucial role in computer vision and remote sensing, to rapidly provide stereo products using inexpensive hardware. Along with the development of deep learning, the Guided Aggregation Network (GA-Net) achieves state-of-the-art performance via the proposed Semi-Global Guided Aggregation layers and reduces the use of costly 3D convolutional layers. To solve the problem of GA-Net requiring large GPU memory consumption, we design a pyramid architecture to modify the model. Starting from a downsampled stereo input, the disparity is estimated and continuously refined through the pyramid levels. Thus, the disparity search is only applied for a small size of stereo pair and then confined within a short residual range for minor correction, leading to highly reduced memory usage and runtime. Tests on close-range, aerial, and satellite data demonstrate that the proposed algorithm achieves significantly higher efficiency (around eight times faster consuming only 20–40% GPU memory) and comparable results with GA-Net on remote sensing data. Thanks to this coarse-to-fine estimation, we successfully process remote sensing datasets with very large disparity ranges, which could not be processed with GA-Net due to GPU memory limitations.

Axonopathy is a pathological feature observed in both Alzheimer’s disease (AD) patients and animal models. However, identifying the temporal and regional progression of axonopathy during AD development remains elusive. Using the fluorescence micro-optical sectioning tomography system, we acquired whole-brain datasets in the early stage of 5xFAD/Thy1-GFP-M mice. We reported that among GFP labeled axons, GFP-positive axonopathy first formed in the lateral septal nucleus, subiculum, and medial mammillary nucleus. The axonopathy further increased in most brain regions during aging. However, most of the axonopathic varicosities disappeared significantly in the medial mammillary nucleus after 8 weeks old. Continuous three-dimensional datasets showed that axonopathy in the medial mammillary nucleus was mainly located on axons from hippocampal GFP-positive neurons. Using the rabies viral tracer in combination with immunohistochemistry, we found that axons in the medial mammillary nucleus from the subiculum were susceptible to lesions that prior to the occurrence of behavioral disorders. In conclusion, we created an early-stage spatiotemporal map of axonopathy in 5xFAD/Thy1-GFP-M mice and identified specific neural circuits which are vulnerable to axon lesions in an AD mouse model. These findings underline the importance of early interventions for AD, and may contribute to the understanding of its progression and its early symptom treatment.

Long-range neuronal circuits play an important role in motor and sensory information processing. Determining direct synaptic inputs of excited and inhibited neurons is important for understanding the circuit mechanisms involved in regulating movement. Here, we used the monosynaptic rabies tracing technique, combined with fluorescent micro-optical sectional tomography, to characterize the brain-wide input to the motor cortex (MC). The whole brain dataset showed that the main excited and inhibited neurons in the MC received inputs from similar brain regions with a quantitative difference. With 3D reconstruction we found that the distribution of input neurons, that target the primary and secondary MC, had different patterns. In the cortex, the neurons projecting to the primary MC mainly distributed in the lateral and anterior portion, while those to the secondary MC distributed in the medial and posterior portion. The input neurons in the subcortical areas also showed the topographic shift model, as in the thalamus, the neurons distributed as outer and inner shells while the neurons in the claustrum and amygdala were in the ventral and dorsal part, respectively. These results lay the anatomical foundation to understanding the organized pattern of motor circuits and the functional differences between the primary and secondary MC.

Semi-Global Matching (SGM) approximates a 2D Markov Random Field (MRF) via multiple 1D scanline optimizations, which serves as a good trade-off between accuracy and efficiency in dense matching. Nevertheless, the performance is limited due to the simple summation of the aggregated costs from all 1D scanline optimizations for the final disparity estimation. SGM-Forest improves the performance of SGM by training a random forest to predict the best scanline according to each scanline’s disparity proposal. The disparity estimated by the best scanline acts as reference to adaptively adopt close proposals for further post-processing. However, in many cases more than one scanline is capable of providing a good prediction. Training the random forest with only one scanline labeled may limit or even confuse the learning procedure when other scanlines can offer similar contributions. In this paper, we propose a multi-label classification strategy to further improve SGM-Forest. Each training sample is allowed to be described by multiple labels (or zero label) if more than one (or none) scanline gives a proper prediction. We test the proposed method on stereo matching datasets, from Middlebury, ETH3D, EuroSDR image matching benchmark, and the 2019 IEEE GRSS data fusion contest. The result indicates that under the framework of SGM-Forest, the multi-label strategy outperforms the single-label scheme consistently.

Building extraction from remote sensing data plays an important role in urban planning, disaster management, navigation, updating geographic databases, and several other geospatial applications [...]

Resin embedding has been widely used for precise imaging of fluorescently labeled biological samples with optical and electron microscopy. The low preservation rate of fluorescence, especially for red fluorescent proteins, has limited the application of resin embedding in multifluorescent protein-labeled samples. Here, we optimized the embedding method to retain the intensity of multiple fluorescent proteins during resin embedding. By reducing the polymerization temperature from 50 to 35°C and adding a fluorescent protein protection reagent during the embedding process, we successfully increased the fluorescence preservation rate by nearly twofold for red fluorescent proteins, including tdTomato, mCherry, and DsRed. Meanwhile, the background fluorescence decreased significantly in the optimized embedding method. This method is suitable not only for red fluorescent protein-labeled samples but also for blue (BFP) and green fluorescent protein (GFP)-labeled samples. We embedded brains labeled with BFP, DsRed, and GFP via AAV and rabies virus and acquired the distribution of input neurons to different cortical areas. With GFP/tdTomato double-labeled samples in resin, we obtained the cholinergic projectome of the pedunculopontine tegmental nucleus (PPTg) and the distribution of cholinergic neurons at single-neuron resolution in the whole brain simultaneously. Input cholinergic terminals from the PPTg were found to innervate the cholinergic soma and fiber in the neocortex, basal forebrain and brainstem, indicating that local cholinergic neurons received long-range cholinergic modulation from the midbrain. Our optimized method is useful for embedding multicolor fluorescent protein-labeled samples to acquire multidimensional structural information on neural circuits at single-neuron resolution in the whole brain.

Large-scale events represent a special challenge for crisis management. To ensure that participants can enjoy an event safely and carefree, it must be comprehensively prepared and attentively monitored. Remote sensing can provide valuable information to identify potential risks and take appropriate measures in order to prevent a disaster, or initiate emergency aid measures as quickly as possible in the event of an emergency. Especially, three-dimensional (3D) information that is derived using photogrammetry can be used to analyze the terrain and map existing structures that are set up at short notice. Using aerial imagery acquired during a German music festival in 2016 and the celebration of the German Protestant Church Assembly of 2017, the authors compare two-dimensional (2D) and novel fusion-based 3D change detection methods, and discuss their suitability for supporting large-scale events during the relevant phases of crisis management. This study serves to find out what added value the use of 3D change information can provide for on-site crisis management. Based on the results, an operational, fully automatic processor for crisis management operations and corresponding products for end users can be developed.

Superpixels, as a state-of-the-art segmentation paradigm, have recently been widely used in computer vision and pattern recognition. Despite the effectiveness of these algorithms, there are still many limitations and challenges dealing with Very High-Resolution (VHR) satellite images especially in complex urban scenes. In this paper, we develop a superpixel algorithm as a modified edge-based version of Simple Linear Iterative Clustering (SLIC), which is here called ESLIC, compatible with VHR satellite images. Then, based on the modified properties of generated superpixels, a heuristic multi-scale approach for building extraction is proposed, based on the stereo satellite imagery along with the corresponding Digital Surface Model (DSM). First, to generate the modified superpixels, an edge-preserving term is applied to retain the main building boundaries and edges. The resulting superpixels are then used to initially refine the stereo-extracted DSM. After shadow and vegetation removal, a rough building mask is obtained from the normalized DSM, which highlights the appropriate regions in the image, to be used as the input of a multi-scale superpixel segmentation of the proper areas to determine the superpixels inside the building. Finally, these building superpixels with different scales are integrated and the output is a unified building mask. We have tested our methods on building samples from a WorldView-2 dataset. The results are promising, and the experiments show that superpixels generated with the proposed ESLIC algorithm are more adherent to the building boundaries, and the resulting building mask retains urban object shape better than those generated with the original SLIC algorithm.

Digital surface models (DSMs) derived from spaceborne and airborne sensors enable the monitoring of the vertical structures for forests in large areas. Nevertheless, due to the lack of an objective performance assessment for this task, it is difficult to select the most appropriate data source for DSM generation. In order to fill this gap, this paper performs change detection analysis including forest decrease and tree growth. The accuracy of the DSMs is evaluated by comparison with measured tree heights from inventory plots (field data). In addition, the DSMs are compared with LiDAR data to perform a pixel-wise quality assessment. DSMs from four different satellite stereo sensors (ALOS/PRISM, Cartosat-1, RapidEye and WorldView-2), one satellite InSAR sensor (TanDEM-X), two aerial stereo camera systems (HRSC and UltraCam) and two airborne laser scanning datasets with different point densities are adopted for the comparison. The case study is a complex central European temperate forest close to Traunstein in Bavaria, Germany. As a major experimental result, the quality of the DSM is found to be robust to variations in image resolution, especially when the forest density is high. The forest decrease results confirm that besides aerial photogrammetry data, very high resolution satellite data, such as WorldView-2, can deliver results with comparable quality as the ones derived from LiDAR, followed by TanDEM-X and Cartosat DSMs. The quality of the DSMs derived from ALOS and Rapid-Eye data is lower, but the main changes are still correctly highlighted. Moreover, the vertical tree growth and their relationship with tree height are analyzed. The major tree height in the study site is between 15 and 30 m and the periodic annual increments (PAIs) are in the range of 0.30–0.50 m.

The intentional damage to local Cultural Heritage sites carried out in recent months by the Islamic State have received wide coverage from the media worldwide. Earth Observation data provide important information to assess this damage in such non-accessible areas, and automated image processing techniques will be needed to speed up the analysis if a fast response is desired. This paper shows the first results of applying fast and robust change detection techniques to sensitive areas, based on the extraction of textural information and robust differences of brightness values related to pre- and post-disaster satellite images. A map highlighting potentially damaged buildings is derived, which could help experts at timely assessing the damages to the Cultural Heritage sites of interest. Encouraging results are obtained for two archaeological sites in Syria and Iraq.