Search

Due to the powerful ability of data fitting, deep neural networks have been applied in a wide range of applications in many key areas. However, in recent years, it was found that some adversarial samples easily fool the deep neural networks. These input samples are generated by adding a few small perturbations based on the original sample, making a very significant influence on the decision of the target model in the case of not being perceived. Image segmentation is one of the most important technologies in the medical image and automatic driving field. This paper mainly explores the security of deep neural network models based on the image segmentation tasks. Two lightweight image segmentation models on the embedded device suffered from the white-box attack by using local perturbations and universal perturbations. The perturbations are generated indirectly by a noise function and an intermediate variable so that the gradient of pixels can be propagated unlimitedly. Through experiments, we find that different models have different blind spots, and the adversarial samples trained for a single model have no transferability. In the end, multiple models are attacked by our joint learning. Finally, under the constraint of low perturbation, most of the pixels in the attacked area have been misclassified by both lightweight models. The experimental result shows that the proposed adversary is more likely to affect the performance of the segmentation model compared with the FGSM. This work was supported in part by the National Natural Science Foundation of China under Grant 61772387, in part by the Fundamental Research Funds of Ministry of Education and China Mobile under Grant MCM20170202, in part by the National Natural Science Foundation of Shaanxi Province under Grant 2019ZDLGY03-03, and in part by the ISN State Key Laboratory.

At present, the cleaning robots on market generally have the defects of simple operation mode and weak intelligence. In order to improve the intelligent degree and operation ability of cleaning robots, this paper proposes a decision method for cleaning robot’s operation mode. Firstly, use the hierarchical expression ability of deep network to obtain the attributes of garbage such as state, shape, distribution, size and so on. Then the causal relationship between the attributes and the operation modes can be built by using joint learning of association attributes with depth network model and causal inference. Based on this, a fuzzy inference network for operation mode decision is designed. With the help of causal analysis, the structure of the decision model is greatly simplified. Compared with conventional fuzzy neural networks, the total parameters of the model are reduced by 2 / 3. The method proposed in this paper imitates the way that human dispose of different types of garbage and has good interpretability. The experimental results verify the effectiveness of the proposed method.

International audience; The objective of this paper is to investigate techniques for learning Fully Connected Network (FCN) models in a lifting based image coding scheme. More precisely, based on a 2D non separable lifting structure composed of three FCN-based prediction stages followed by an FCN-based update one, we first propose to resort to an p loss function, with p ∈ {1, 2}, to learn the three FCN prediction models. While the latter are separately learned in the first approach, a novel joint learning approach is then developed by minimizing a weighted p loss function related to the global prediction error. Experimental results, carried out on the standard Challenge Learned Image Compression (CLIC) dataset, show the benefits of the proposed techniques in terms of rate-distortion performance.

Intent classification and slot filling are two essential tasks for spoken language understanding. Recently, joint learning has been shown to be effective for the two tasks. However, most joint learning methods only consider joint learning using shared parameters on the surface level rather than the semantic level, and these methods suffer from small-scale human-labeled training data, resulting in poor generalization capabilities, especially for rare words. In this paper, we propose a novel encoder-decoder framework based multi-task learning model, which conducts joint training for intent classification and slot filling tasks. For the encoder of our model, we encode the input sequence as context representations using bidirectional encoder representation from transformers (BERT). For the decoder, we implement two-stage decoder process in our model. In the first stage, we use an intent classification decoder to detect the user's intent. In the second stage, we leverage the intent contextual information into the slot filling decoder to predict the semantic concept tags for each word. We conduct experiments on three popular benchmark datasets: ATIS, Snips and Facebook multilingual task-oriented datasets. The experimental results show that our proposed model outperforms the state-of-the-art approaches and achieves new state-of-the-art results on both three datasets.

International audience; Extensive attention has been widely paid to enhance the spatial resolution of hyperspectral (HS) images with the aid of multispectral (MS) images in remote sensing. However, the ability in the fusion of HS and MS images remains to be improved, particularly in large-scale scenes, due to the limited acquisition of HS images. Alternatively, we super-resolve MS images in the spectral domain by the means of partially overlapped HS images, yielding a novel and promising topic: spectral superresolution (SSR) of MS imagery. This is challenging and less investigated task due to its high ill-posedness in inverse imaging. To this end, we develop a simple but effective method, called joint sparse and low-rank learning (J-SLoL), to spectrally enhance MS images by jointly learning low-rank HS-MS dictionary pairs from overlapped regions. J-SLoL infers and recovers the unknown HS signals over a larger coverage by sparse coding on the learned dictionary pair. Furthermore, we validate the SSR performance on three HS-MS data sets (two for classification and one for unmixing) in terms of reconstruction, classification, and unmixing by comparing with several existing state-of-the-art baselines, showing the effectiveness and superiority of the proposed J-SLoL algorithm. Furthermore, the codes and data sets will be available at https://github.com/danfenghong/IEEE_TGRS_J-SLoL, contributing to the remote sensing (RS) community.

Single-cell RNA-sequencing (scRNA-seq) explores the transcriptome of genes at cell level, which sheds light on revealing the heterogeneity and dynamics of cell populations. Advances in biotechnologies make it possible to generate scRNA-seq profiles for large-scale cells, requiring effective and efficient clustering algorithms to identify cell types and informative genes. Although great efforts have been devoted to clustering of scRNA-seq, the accuracy, scalability and interpretability of available algorithms are not desirable. In this study, we solve these problems by developing a joint learning algorithm [a.k.a. joints sparse representation and clustering (jSRC)], where the dimension reduction (DR) and clustering are integrated. Specifically, DR is employed for the scalability and joint learning improves accuracy. To increase the interpretability of patterns, we assume that cells within the same type have similar expression patterns, where the sparse representation is imposed on features. We transform clustering of scRNA-seq into an optimization problem and then derive the update rules to optimize the objective of jSRC. Fifteen scRNA-seq datasets from various tissues and organisms are adopted to validate the performance of jSRC, where the number of single cells varies from 49 to 110 824. The experimental results demonstrate that jSRC significantly outperforms 12 state-of-the-art methods in terms of various measurements (on average 20.29% by improvement) with fewer running time. Furthermore, jSRC is efficient and robust across different scRNA-seq datasets from various tissues. Finally, jSRC also accurately identifies dynamic cell types associated with progression of COVID-19. The proposed model and methods provide an effective strategy to analyze scRNA-seq data (the software is coded using MATLAB and is free for academic purposes; https://github.com/xkmaxidian/jSRC).

Language-based person search retrieves images of a target person using natural language description and is a challenging fine-grained cross-modal retrieval task. A novel hybrid attention network is proposed for the task. The network includes the following three aspects: First, a cubic attention mechanism for person image, which combines cross-layer spatial attention and channel attention. It can fully excavate both important midlevel details and key high-level semantics to obtain better discriminative fine-grained feature representation of a person image. Second, a text attention network for language description, which is based on bidirectional LSTM (BiLSTM) and self-attention mechanism. It can better learn the bidirectional semantic dependency and capture the key words of sentences, so as to extract the context information and key semantic features of the language description more effectively and accurately. Third, a cross-modal attention mechanism and a joint loss function for cross-modal learning, which can pay more attention to the relevant parts between text and image features. It can better exploit both the cross-modal and intra-modal correlation and can better solve the problem of cross-modal heterogeneity. Extensive experiments have been conducted on the CUHK-PEDES dataset. Our approach obtains higher performance than state-of-the-art approaches, demonstrating the advantage of the approach we propose.

Human activity recognition has become essential to a wide range of applications, such as smart home monitoring, health-care, surveillance. However, it is challenging to deliver a sufficiently robust human activity recognition system from raw sensor data with noise in a smart environment setting. Moreover, imbalanced human activity datasets with less frequent activities create extra challenges for accurate activity recognition. Deep learning algorithms have achieved promising results on balanced datasets, but their performance on imbalanced datasets without explicit algorithm design cannot be promised. Therefore, we aim to realise an activity recognition system using multi-modal sensors to address the issue of class imbalance in deep learning and improve recognition accuracy. This paper proposes a joint diverse temporal learning framework using Long Short Term Memory and one-dimensional Convolutional Neural Network models to improve human activity recognition, especially for less represented activities. We extensively evaluate the proposed method for Activities of Daily Living recognition using binary sensors dataset. A comparative study on five smart home datasets demonstrate that our proposed approach outperforms the existing individual temporal models and their hybridization. Furthermore, this is particularly the case for minority classes in addition to reasonable improvement on the majority classes of human activities.

In various case studies across Europe the vital role of rural place leadership in enabling a place-based approach to local and regional development has been highlighted, although not always explicitly addressed as such. This paper aims to do so by reviewing the findings from a selection of earlier research projects within a framework of the role of rural leadership in place-based development. Building on the increasing body of literature on place leadership, the review reveals how place leadership in rural areas is performed by varied public, private and civic actors; is able to bridge vested stakes and make new connections; is supportive to joint learning and innovation and an increasing range of bottom-up grassroots initiatives. Effective rural place leadership initiates joint reflection and enforces a collaborative spirit resulting in an expanding spiral of new alliances and new (institutional) arrangements. This underpins the importance of rural place leadership in building collective agency and its capacity to better attune the institutional setting to the specificities of place and thus enhance place-based development.

Context: Hedgerows are typical landscape features of high environmental and cultural value that often have been sacrificed for agricultural intensification and scale enlargement. Objectives: We studied the dynamics of hedgerow quality over time in a case study area renowned for its hedgerow landscapes: South West Devon (UK) answering the following research questions: (1) how does the imperative of scale enlargement affect hedgerow quality? and (2) to what extent can cultural landscape degradation be countered by targeted policies? Methods: We applied an agent-based modeling approach, parameterized with a site specific survey, to explore and discuss outcomes of future landscape change with stakeholders and co-designed preferred scenarios of landscape change during a workshop. Results: Outcomes suggested that in the case-study area, scale enlargement has a negative effect on hedgerow quality when agri-environment scheme subsidies (AES) are low. In contrast, if the level of AES enrollment is high, scale enlargement can have a positive effect on hedgerow quality, as large holders are more likely to enroll for AES. Stakeholders acknowledged the need for agricultural intensification, but at the same time valued biodiversity and environmental value of the landscape in South West Devon. Conclusion: Current AES are able to retain a decent hedgerow quality. With lower AES, scale enlargement can have an invigorative effect on hedgerow quality as land managers of larger farms will be less likely to join AES As an addition to AES, harvesting wood fuel from coppiced hedgerows appears a promising way to incentivize rejuvenating hedgerow management without governmental subsidies.

International audience; In this work, we propose a novel multi-task framework, to learn satellite image pansharpening and segmentation jointly. Our framework is based on the encoder-decoder architecture, where both tasks share the same encoder but each one has its own decoder. We compare our framework against single-task models with different architectures. Results show that our framework outperforms all other approaches in both tasks.

Transformer health index (HI) is a powerful tool for quantifying the overall health of a power transformer, due to the fact that it appraises its condition based on different criteria that are related (often in complex ways) to the long-term degradation factors that cumulatively lead to its end-of-life. Several authors have proposed different approaches to the HI calculation, e.g., analytical expressions, binary logistic regression, fuzzy logic models, support vector machines, and artificial neural networks. This paper proposes using Bayesian "Wide & Deep" machine learning model for the HI calculation, where the wide model part is the Bayesian ordered robust "probit" regression, while the deep part is the Bayesian artificial neural network. Both model parts are trained simultaneously within the Bayesian setting, using the so-called "joint learning" process with a Markov-chain Monte Carlo algorithm. Model is demonstrated using the actual transformer data.

acceptedVersion

The industrial Internet of Things (IoT) can monitor production in real-time by collecting the status of parts on the production line with cameras. It is easy to have bright and dark areas in the same image because of the smooth surfaces of mechanical parts and the unstable light source, which affects semantic segmentation’s performance. This paper proposes a joint learning method to eliminate the influence of illumination on semantic segmentation. Semantic image segmentation and image decomposition are jointly trained in the same model, and the reflectance image is used to guide the semantic segmentation task without the illumination component. Moreover, this paper adopts an enhanced convolution kernel to improve the pixel accuracy and BN fusion to enhance the inference speed, optimizing the model to meet real-time detection needs. In the experiments, a dataset of real gear parts was collected from industrial IoT cameras. The experimental results show that the proposed joint learning approach outperforms the state-of-the-art methods in the task of edge mechanical part detection, with about 4% pixel accuracy improvement.

Brain tissue segmentation from multimodal MRI is a key building block of many neuroimaging analysis pipelines. Established tissue segmentation approaches have, however, not been developed to cope with large anatomical changes resulting from pathology, such as white matter lesions or tumours, and often fail in these cases. In the meantime, with the advent of deep neural networks (DNNs), segmentation of brain lesions has matured significantly. However, few existing approaches allow for the joint segmentation of normal tissue and brain lesions. Developing a DNN for such a joint task is currently hampered by the fact that annotated datasets typically address only one specific task and rely on task-specific imaging protocols including a task-specific set of imaging modalities. In this work, we propose a novel approach to build a joint tissue and lesion segmentation model from aggregated task-specific hetero-modal domain-shifted and partially-annotated datasets. Starting from a variational formulation of the joint problem, we show how the expected risk can be decomposed and optimised empirically. We exploit an upper bound of the risk to deal with heterogeneous imaging modalities across datasets. To deal with potential domain shift, we integrated and tested three conventional techniques based on data augmentation, adversarial learning and pseudo-healthy generation. For each individual task, our joint approach reaches comparable performance to task-specific and fully-supervised models. The proposed framework is assessed on two different types of brain lesions: White matter lesions and gliomas. In the latter case, lacking a joint ground-truth for quantitative assessment purposes, we propose and use a novel clinically-relevant qualitative assessment methodology., MIDL 2019 special issue - Medical Image Analysis

It is highly desirable to predict the progression of Alzheimer's disease (AD) of patients [e.g., to predict conversion of mild cognitive impairment (MCI) to AD], especially longitudinal prediction of AD is important for its early diagnosis. Currently, most existing methods predict different clinical scores using different models, or separately predict multiple scores at different future time points. Such approaches prevent coordinated learning of multiple predictions that can be used to jointly predict clinical scores at multiple future time points. In this paper, we propose a joint learning method for predicting clinical scores of patients using multiple longitudinal prediction models for various future time points. Three important relationships among training samples, features, and clinical scores are explored. The relationship among different longitudinal prediction models is captured using a common feature set among the multiple prediction models at different time points. Our experimental results based on the Alzheimer's disease neuroimaging initiative (ADNI) database shows that our method achieves considerable improvement over competing methods in predicting multiple clinical scores.

Learning networks have the potential to support joint learning and collective innovation processes needed for sustainable urban development. However, systematic analyses of joint learning processes in such networks are often lacking. In this paper, the Interreg project SUN (Sustainable Urban Neighbourhoods) is taken as a case study to analyse learning interactions in a regional network for sustainable urban neighbourhood development in the Euregio Meuse-Rhine. The SUN project provides concerted public and private action, focussing on community engagement, cross-border cooperation, and collective innovation. In this context, joint learning implies the exchange and co-production of knowledge among a diverse group of actors. We first describe the learning network of SUN in terms of the actors involved, the learning activities and tools applied, and the types of knowledge exchanged and co-produced (technical, attitudinal, innovative). These features are further used to characterize a set of typical learning interactions that generated learning across four main knowledge boundaries (territorial, role-based, sectoral, and project boundaries). From this analysis, we identify main factors supporting the joint learning process, and discuss the general applicability and added value of the developed analytical framework. (C) 2012 Elsevier Ltd. All rights reserved.

We present an occlusion-aware unsupervised neural network for jointly learning three low-level vision tasks from monocular videos: depth, optical flow, and camera motion. The system consists of three different predicting sub-networks simultaneously coupled by combined loss terms and is capable of computing each task independently on test samples. Geometric constraints extracted from scene geometry which have traditionally been used in bundle adjustment or pose-graph optimization are formed as various self-supervisory signals during our end-to-end learning approach. Different from prior works, our image reconstruction loss also takes account of optical flow. Moreover, we impose novel 3D flow consistency constraints over the predictions of all the three tasks. By explicitly modeling occlusion and taking utilization of both 2D and 3D geometry relationships, abundant geometric constraints are formed over estimated outputs, enabling the system to capture both low-level representations and high-level cues to infer thinner scene structures. Empirical evaluation on the KITTI dataset demonstrates the effectiveness and improvement of our approach: (1) monocular depth estimation outperforms state-of-the-art unsupervised methods and is comparable to stereo supervised ones, (2) optical flow prediction ranks top among prior works and even beats supervised and traditional ones especially in non-occluded regions, (3) pose estimation outperforms established SLAM systems under comparable input settings with a reasonable margin.

Our system for the CoNLL 2008 shared task uses a set of individual parsers, a set of stand-alone semantic role labellers, and a joint system for parsing and semantic role labelling, all blended together. The system achieved a macro averaged labelled F1- score of 79.79 (WSJ 80.92, Brown 70.49) for the overall task. The labelled attachment score for syntactic dependencies was 86.63 (WSJ 87.36, Brown 80.77) and the labelled F1-score for semantic dependencies was 72.94 (WSJ 74.47, Brown 60.18). Publisher: Coling 2008 Organizing Committee. Pages: 248 - 252.