Search

In clinical medicine, magnetic resonance imaging (MRI) is one of the most important tools for diagnosis, triage, prognosis, and treatment planning. However, MRI suffers from an inherent slow data acquisition process because data is collected sequentially in k-space. In recent years, most MRI reconstruction methods proposed in the literature focus on holistic image reconstruction rather than enhancing the edge information. This work steps aside this general trend by elaborating on the enhancement of edge information. Specifically, we introduce a novel parallel imaging coupled dual discriminator generative adversarial network (PIDD-GAN) for fast multi-channel MRI reconstruction by incorporating multi-view information. The dual discriminator design aims to improve the edge information in MRI reconstruction. One discriminator is used for holistic image reconstruction, whereas the other one is responsible for enhancing edge information. An improved U-Net with local and global residual learning is proposed for the generator. Frequency channel attention blocks (FCA Blocks) are embedded in the generator for incorporating attention mechanisms. Content loss is introduced to train the generator for better reconstruction quality. We performed comprehensive experiments on Calgary-Campinas public brain MR dataset and compared our method with state-of-the-art MRI reconstruction methods. Ablation studies of residual learning were conducted on the MICCAI13 dataset to validate the proposed modules. Results show that our PIDD-GAN provides high-quality reconstructed MR images, with well-preserved edge information. The time of single-image reconstruction is below 5ms, which meets the demand of faster processing., Comment: 33 pages, 13 figures, Applied Intelligence

Clustering data with both numeric and categorical attributes is of great importance as such data are ubiquitous in real-world problems. Multi-view learning approaches have proven to be more effective and having better generalisation ability compared to single-view learning in many problems. However, most of the existing clustering algorithms developed for mixed numeric and categorical data are single-view. In this research, we propose a novel multi-view clustering algorithm based on the k-prototypes (which we term Multi-view K-Prototypes) for clustering mixed data. To the best of our knowledge, our proposed Multi-view K-Prototypes is the first multi-view version of the well-known k-prototypes algorithm. To cluster the mixed data over multiple views, we present a novel representation prototype of cluster centres in the scenario of multiple views, and we also devise formulas for updating the cluster centres over each view. Then we propose the concept of consensus cluster centres to output the final clustering result. Finally, we carried out a series of experiments on four benchmark datasets to assess the performance of the proposed Multi-view K-Prototypes clustering. Experimental results show that the Multi-view K-Prototypes algorithm outperforms the seven state-of-the-art algorithms in most cases.

This paper proposes a weighted hybrid multi-view fusion method for the semi-supervised classification problem. Instead of getting access to the features from different views directly, this method utilizes the square losses of the multi-view classifiers to exploit the between-view relationship, which preserves the privacy of data. Considering the different prediction capability of classifiers on multiple views, an objective function with the constraint of rank consistency is constructed to weight view-specific learners adaptively, where the constraint makes each view-specific learner improve its performance by exploring the predicted results of other learners. Furthermore, an iterative algorithm based on the Variant Alternating Splitting Augmented Lagrangian Method (VASALM) and the quadratic programming method is developed to optimize the objective function. Experimental results on different real-world datasets demonstrate the effectiveness of the proposed method for multi-view learning. The experiments also analyze parameter sensitivity and convergency of the optimization algorithm. Refereed/Peer-reviewed

Nowadays, the time lag between vulnerability discovery and the timely remediation of the vulnerability is extremely important to the current state of cybersecurity. Unfortunately, the silent security patch presents a significant challenge. Despite related work having been conducted in this area, the patch identification lacks interpretability. To solve this problem, this paper first proposes a trusted multi-view security patch identification system called TMVDPatch. The system obtains evidence from message commit and code diff views respectively, and models the uncertainty of each view based on the D-S evidence theory, thereby providing credible and interpretable security patch identification results. On this basis, this paper performs weighted training on the original evidence based on the grey relational analysis method to improve the ability to make credible decisions based on multi-views. Experimental results show that the multi-view learning method exhibits excellent capabilities in terms of the complementary information provided by control dependency and data dependency, and the model shows strong robustness across different hyperparameter settings. TMVDPatch outperforms other models in all evaluation metrics, achieving an accuracy of 85.29% and a F1 score of 0.9001, clearly verifying the superiority of TMVDPatch in terms of accuracy, scientificity, and reliability.

The ability to build more robust clustering from many clustering models with different solutions is relevant in scenarios with privacy-preserving constraints, where data features have a different nature or where these features are not available in a single computation unit. Additionally, with the booming number of multi-view data, but also of clustering algorithms capable of producing a wide variety of representations for the same objects, merging clustering partitions to achieve a single clustering result has become a complex problem with numerous applications. To tackle this problem, we propose a clustering fusion algorithm that takes existing clustering partitions acquired from multiple vector space models, sources, or views, and merges them into a single partition. Our merging method relies on an information theory model based on Kolmogorov complexity that was originally proposed for unsupervised multi-view learning. Our proposed algorithm features a stable merging process and shows competitive results over several real and artificial datasets in comparison with other state-of-the-art methods that have similar goals.

This paper considers unsupervised dimensionality reduction of multi-view data, where locality preserving canonical correlation analysis and a new locality-preserving canonical correlation analysis are two typical effective methods. However, they ignore the global structure while considering the local structure of data, and are sensitive to noises because of the relationship of neighbors based on the Euclidean distance. In this paper, we propose a novel multi-view dimensionality reduction method: multiset canonical correlations analysis based on low-rank representation. Our model introduces the cross-view similarity matrix to consider the correlation of all different points in cross views, which makes it not only preserve the local structure but also the global structure of data. And the cross-view similarity matrix is constructed by using low-rank representation, which can make the model more robust. In addition, a parameter $\beta $ is introduced to adjust the importance of the correlation of different sample points, enhancing the generalization ability for different datasets. Experiments on four multi-view datasets show our proposed method has better performance than the related methods.

This paper represents a semi-supervised learning framework, which integrates multi-view learning, extreme learning machine (ELM) and graph-based semi-supervised learning. The aim is to expand the scope of adaptation of non-negative sparse graph (NNSG) framework, under a multi-view condition and a non-linear relationship. The proposed multi-view learning method will be adaptive since when data is single-view the framework will degenerate into an embedded framework for NNSG framework. The proposed ELM method also will be adaptive since the number of hidden layer neuron will change with different number of input and output layer neuron. The combination of both proposed methods outperforms traditional graph-based semi-supervised learning, such as flexible manifold embedding (FME) and NNSG framework, which can not establish an affinity matrix for multi-view and can not establish a non-liner model for unknown data. Unlike traditional graph-based semi-supervised learning methods, which only can label propagation and build linear regression models for single or multi-view data, our proposed method has an obvious advantage that is applicable to any single or multi-view data, and builds linear or non-linear models. We provides extensive experiments on four public database in order to evaluate the performance of the proposed method. These experiments demonstrate significant improvement over the state-of-the-art algorithms in label propagation and processing of new data.

In recent years, many studies have been carried out to detect clouds on remote sensing images. Due to the complex terrain, the variety of clouds, the density, and content of clouds are various, and the current model has difficulty accurately detecting the cloud in the image. In our strategy, a multi-view data training set based on super pixel is constructed. View A uses multi-level network to extract the boundary, texture, and deep abstract feature of super pixels. View B is the statistical feature of the three channels of the image. Privilege information View P contains the cloud content of super pixels and the tag status of adjacent super pixels. Finally, we propose a cloud detection method for remote sensing image classification based on multi-view support vector machine (SVM). The proposed method is tested on images of different terrain and cloud distribution in GF-1_WHU and Cloud-38 remote sensing datasets. Visual performance and quantitative analysis show that the method has excellent cloud detection performance.

Material classification is similar to texture classification and consists in predicting the material class of a surface in a color image, such as wood, metal, water, wool, or ceramic. It is very challenging because of the intra-class variability. Indeed, the visual appearance of a material is very sensitive to the acquisition conditions such as viewpoint or lighting conditions. Recent studies show that deep convolutional neural networks (CNNs) clearly outperform hand-crafted features in this context but suffer from a lack of data for training the models. In this paper, we propose two contributions to cope with this problem. First, we provide a new material dataset with a large range of acquisition conditions so that CNNs trained on these data can provide features that can adapt to the diverse appearances of the material samples encountered in real-world. Second, we leverage recent advances in multi-view learning methods to propose an original architecture designed to extract and combine features from several views of a single sample. We show that such multi-view CNNs significantly improve the performance of the classical alternatives for material classification.

Background: The brain magnetic resonance imaging (MRI) image segmentation method mainly refers to the division of brain tissue, which can be divided into tissue parts such as white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). The segmentation results can provide a basis for medical image registration, 3D reconstruction, and visualization. Generally, MRI images have defects such as partial volume effects, uneven grayscale, and noise. Therefore, in practical applications, the segmentation of brain MRI images has difficulty obtaining high accuracy.Materials and Methods: The fuzzy clustering algorithm establishes the expression of the uncertainty of the sample category and can describe the ambiguity brought by the partial volume effect to the brain MRI image, so it is very suitable for brain MRI image segmentation (B-MRI-IS). The classic fuzzy c-means (FCM) algorithm is extremely sensitive to noise and offset fields. If the algorithm is used directly to segment the brain MRI image, the ideal segmentation result cannot be obtained. Accordingly, considering the defects of MRI medical images, this study uses an improved multiview FCM clustering algorithm (IMV-FCM) to improve the algorithm’s segmentation accuracy of brain images. IMV-FCM uses a view weight adaptive learning mechanism so that each view obtains the optimal weight according to its cluster contribution. The final division result is obtained through the view ensemble method. Under the view weight adaptive learning mechanism, the coordination between various views is more flexible, and each view can be adaptively learned to achieve better clustering effects.Results: The segmentation results of a large number of brain MRI images show that IMV-FCM has better segmentation performance and can accurately segment brain tissue. Compared with several related clustering algorithms, the IMV-FCM algorithm has better adaptability and better clustering performance.

Background: The brain magnetic resonance imaging (MRI) image segmentation method mainly refers to the division of brain tissue, which can be divided into tissue parts such as white matter (WM), gray matter (GM), and cerebrospinal fluid (CSF). The segmentation results can provide a basis for medical image registration, 3D reconstruction, and visualization. Generally, MRI images have defects such as partial volume effects, uneven grayscale, and noise. Therefore, in practical applications, the segmentation of brain MRI images has difficulty obtaining high accuracy. Materials and Methods: The fuzzy clustering algorithm establishes the expression of the uncertainty of the sample category and can describe the ambiguity brought by the partial volume effect to the brain MRI image, so it is very suitable for brain MRI image segmentation (B-MRI-IS). The classic fuzzy c-means (FCM) algorithm is extremely sensitive to noise and offset fields. If the algorithm is used directly to segment the brain MRI image, the ideal segmentation result cannot be obtained. Accordingly, considering the defects of MRI medical images, this study uses an improved multiview FCM clustering algorithm (IMV-FCM) to improve the algorithm’s segmentation accuracy of brain images. IMV-FCM uses a view weight adaptive learning mechanism so that each view obtains the optimal weight according to its cluster contribution. The final division result is obtained through the view ensemble method. Under the view weight adaptive learning mechanism, the coordination between various views is more flexible, and each view can be adaptively learned to achieve better clustering effects. Results: The segmentation results of a large number of brain MRI images show that IMV-FCM has better segmentation performance and can accurately segment brain tissue. Compared with several related clustering algorithms, the IMV-FCM algorithm has better adaptability and better clustering performance.

Background: Traditionally, machine learning algorithms have been simply applied for software defect prediction by considering single-view data, meaning the input data contains a single feature vector. Nevertheless, different software engineering data sources may include multiple and partially independent information, which makes the standard single-view approaches ineffective. Objective: In order to overcome the single-view limitation in the current studies, this article proposes the usage of a multi-view learning method for software defect classification problems. Method: The Multi-View k-Nearest Neighbors (MVKNN) method was used in the software engineering field. In this method, first, base classifiers are constructed to learn from each view, and then classifiers are combined to create a robust multi-view model. Results: In the experimental studies, our algorithm (MVKNN) is compared with the standard k-nearest neighbors (KNN) algorithm on 50 datasets obtained from different software bug repositories. The experimental results demonstrate that the MVKNN method outperformed KNN on most of the datasets in terms of accuracy. The average accuracy values of MVKNN are 86.59%, 88.09%, and 83.10% for the NASA MDP, Softlab, and OSSP datasets, respectively. Conclusion: The results show that using multiple views (MVKNN) can usually improve classification accuracy compared to a single-view strategy (KNN) for software defect prediction.

Metric learning is a class of efficient algorithms for EEG signal classification problem. Usually, metric learning method deals with EEG signals in the single view space. To exploit the diversity and complementariness of different feature representations, a new auto-weighted multi-view discriminative metric learning method with Fisher discriminative and global structure constraints for epilepsy EEG signal classification called AMDML is proposed to promote the performance of EEG signal classification. On the one hand, AMDML exploits the multiple features of different views in the scheme of the multi-view feature representation. On the other hand, considering both the Fisher discriminative constraint and global structure constraint, AMDML learns the discriminative metric space, in which the intraclass EEG signals are compact and the interclass EEG signals are separable as much as possible. For better adjusting the weights of constraints and views, instead of manually adjusting, a closed form solution is proposed, which obtain the best values when achieving the optimal model. Experimental results on Bonn EEG dataset show AMDML achieves the satisfactory results.

Recognition of motor imagery intention is one of the hot current research focuses of brain-computer interface (BCI) studies. It can help patients with physical dyskinesia to convey their movement intentions. In recent years, breakthroughs have been made in the research on recognition of motor imagery task using deep learning, but if the important features related to motor imagery are ignored, it may lead to a decline in the recognition performance of the algorithm. This paper proposes a new deep multi-view feature learning method for the classification task of motor imagery electroencephalogram (EEG) signals. In order to obtain more representative motor imagery features in EEG signals, we introduced a multi-view feature representation based on the characteristics of EEG signals and the differences between different features. Different feature extraction methods were used to respectively extract the time domain, frequency domain, time-frequency domain and spatial features of EEG signals, so as to made them cooperate and complement. Then, the deep restricted Boltzmann machine (RBM) network improved by t-distributed stochastic neighbor embedding(t-SNE) was adopted to learn the multi-view features of EEG signals, so that the algorithm removed the feature redundancy while took into account the global characteristics in the multi-view feature sequence, reduced the dimension of the multi-visual features and enhanced the recognizability of the features. Finally, support vector machine (SVM) was chosen to classify deep multi-view features. Applying our proposed method to the BCI competition IV 2a dataset we obtained excellent classification results. The results show that the deep multi-view feature learning method further improved the classification accuracy of motor imagery tasks.

In multi-view multi-label learning, each object is represented by multiple data views, and belongs to multiple class labels simultaneously. Generally, all the data views have a contribution to the multi-label learning task, but their contributions are different. Besides, for each data view, each class label is only associated with a subset data features, and different features have different contributions to each class label. In this paper, we propose a novel framework VLSF for multi-view multi-label learning, i.e., multi-view multi-label learning with View-Label-Specific Features. Specifically, we first learn a low dimensional label-specific data representation for each data view and construct a multi-label classification model based on it by exploiting label correlations and view consensus, and learn the contribution weight of each data view to multi-label learning task for all the class labels jointly. Then, the final prediction can be made by combing the prediction results of all the classifiers and the learned contribution weights. The extensive comparison experiments with the state-of-the-art approaches manifest the effectiveness of the proposed method VLSF.

Graph-based multi-view feature learning methods learn a low-dimensional embedding of the data by modeling the affinity correlations with a graph to reduce the dimension. However, the learned low-dimensional representation relies on a fixed graph that is potentially inaccurate and unreliable. Besides, the graph construction and the projection matrix leaning are separated into two independent processes. To tackle the problems, we propose a robust unsupervised multi-view feature learning method with a dynamic graph. The dynamic graph structure is constructed adaptively and the robust projection matrix is learned simultaneously. Specifically, we adaptively learn a dynamic graph which captures the intrinsic multiple view-specific relations of samples. Robust projection matrix learning suppresses the adverse noises and preserves the intrinsic graph structure. Moreover, the assigned weights are learned automatically for each view without any extra parameter. We finally develop an efficient alternative optimization algorithm to solve the objective formulation. The extensive experiments conducted on several multi-view datasets demonstrate the effectiveness of our proposed method.

Grading of glioma is crucial for treatment decision making as well as prognostic assessments. In clinical routines, radiologists grade gliomas with multiple complementary magnetic resonance imaging (MRI) sequences, which is yet challenging for glioma prediction models. In this paper, we take full advantages of four commonly used MRI sequences to propose non-invasive grading of glioma based on a variant of maximum entropy discrimination (MED) and decision tree. First, radiomics features calculation is, respectively, performed on T1-weighted imaging, T2-weighted imaging, fluid attenuation inversion recovery imaging, and contrast-enhanced T1-weighted imaging. Then, radiomics features are integrated to build a glioma prediction model named four-sequence MED (FSMED) according to the assumption that the classification margin of different sequences is consistent. Finally, we propose a multi-MED decision tree (MMEDT) model to obtain the grading of gliomas based on the output of FSMED and the results of MED on each sequence. Validation experiments are conducted on a data set collected from Henan Provincial People's Hospital (GliomaHPPH2018) and Multimodal Brain Tumor Image Segmentation Benchmark 2017 (BraTS2017). The results of these two data sets demonstrate the high-prediction performance of our method. The average areas under the curve (AUC) of MMEDT are 0.9119, 0.8184, and 0.9084 for GliomaHPPH2018, BraTS2017, and their merged set, respectively, with the corresponding average sensitivities of 92.55%, 87.85%, and 87.91%, and average specificities of 92.57%, 81.36%, and 87.39%.

Purpose: Automated segmentation of abnormal tissues in medical images is considered as an essential part of those computer-aided detection and diagnosis systems which analyze medical images. However, automated segmentation of abnormalities is a challenging task due to the limitations of imaging technologies and complex structure of abnormalities, including low contrast between normal and abnormal tissues, shape diversity, appearance inhomogeneity, and the vague boundaries of abnormalities. Therefore, more intelligent segmentation techniques are required to tackle these challenges. Materials and Methods: In this study, a method, which is called MMTDNN, is proposed to segment and detect medical image abnormalities. MMTDNN, as a multi-view learning machine, utilizes convolutional neural networks in a massive training strategy. Moreover, the proposed method has four phases of preprocessing, view generation, pixel-level segmentation, and post-processing. The International Symposium on Biomedical Imaging (ISBI)-2016 dataset is used for the evaluation of the proposed method. Results: The performance of the proposed method has been evaluated on the task of skin lesion segmentation as one of the challenging applications of abnormal tissue segmentation. Both qualitative and quantitative results demonstrate outstanding performance. Meanwhile, the accuracy of 0.973, the Jaccard index of 0.876, and the Dice similarity coefficient of 0.931 have been achieved. Conclusion: In conclusion, the experimental result demonstrates that the proposed method outperforms stateof-the-art methods of skin lesion segmentation.

© 2017 Elsevier B.V. In multi-view clustering, datasets are comprised of different representations of the data, or views. Although each view could individually be used, exploiting information from all views together could improve the cluster quality. In this paper a new model Multi-View Kernel Spectral Clustering (MVKSC) is proposed that performs clustering when two or more views are available. This model is formulated as a weighted kernel canonical correlation analysis in a primal-dual optimization setting typical of Least Squares Support Vector Machines (LS-SVM). The primal model includes, in particular, a coupling term, which enforces the clustering scores corresponding to the different views to align. Because of the out-of-sample extension, this model is easily applied to large-scale datasets. The performance of the proposed model is shown on synthetic and real-world datasets, as well as on some large-scale datasets. Experimental comparisons with a number of other methods show that using multiple views improves the clustering results and that the proposed method is competitive with other state-of-the-art algorithms in terms of clustering accuracy and runtime. Especially on the large-scale datasets the advantage of the proposed method is clearly shown, as it is able to handle larger datasets than the other state-of-the-art algorithms. ispartof: An International Journal on Information Fusion vol:44 pages:46-56 status: published

In order to more accurately recognize and understand the working condition of sucker-rod pumping wells so as to maximally reduce the cost and increase the profit, a large amount of data has been collected during oil production with sucker-rod pumping wells. In view of the sucker-rod pumping production system in big data and IOT (Internet of things) of oil-gas production, to solve the limitations in the existing working condition recognition research and further improve the recognition accuracy and practicality with fewer labelled working condition samples by utilizing the measured parameters from multiple information sources effectively, in this paper, a novel working condition recognition method based on Hessian-regularized weighted multi-view canonical correlation analysis is proposed. Firstly, the features of the measured ground dynamometer cards, electrical power, wellhead temperature and wellhead pressure data are extracted as four different feature views based on the prior information, empirical knowledge and mechanism analysis. Then a model based on Hessian-regularized weighted multi-view canonical correlation analysis and cosine nearest neighbour multi-classification algorithm is established. The proposed method is applied to the recognition of eleven kinds of working conditions from sixty sucker-rod pumping wells in a certain block in Shengli Oilfield, China. In the case where there are small number of labelled training samples, based on cosine nearest neighbour classification method, the recognition rates are increased by 3.44% and 1.5% compared with traditional recognition methods based on measured ground dynamometer cards and electrical power data, respectively. In contrast to methods based on traditional multi-sources of feature connection, multi-view canonical correlation analysis as well as the unweighted Hessian-regularized multi-view canonical correlation analysis, the recognition rates are increased by 4.46%, 2.21% and 1.62%, respectively.

Since their beginnings, social networks have affected the way people communicate and interact with each other. The continuous growing and pervasive use of social media offers interesting research opportunities for analysing the behaviour and interactions of users. Nowadays, interactions are not only limited to social relations, but also to reading and writing activities. Thus, multiple and complementary information sources are available for characterising users and their activities. One task that could benefit from the integration of those multiple sources is community detection. However, most techniques disregard the effect of information aggregation and continue to focus only on one aspect: the topological structure of networks. This paper focuses on how to integrate social and content-based information originated in social networks for improving the quality of the detected communities. A technique for integrating both the multiple information sources and the semantics conveyed by asymmetric relations is proposed and extensively evaluated on two real-world datasets. Experimental evaluation confirmed the differentiated impact that each information source has on the quality of the detected communities, and shed some light on how to improve such quality by combining both social and content-based information. Fil: Tommasel, Antonela. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tandil. Instituto Superior de Ingeniería del Software. Universidad Nacional del Centro de la Provincia de Buenos Aires. Instituto Superior de Ingeniería del Software; Argentina Fil: Godoy, Daniela Lis. Consejo Nacional de Investigaciones Científicas y Técnicas. Centro Científico Tecnológico Conicet - Tandil. Instituto Superior de Ingeniería del Software. Universidad Nacional del Centro de la Provincia de Buenos Aires. Instituto Superior de Ingeniería del Software; Argentina

International audience; Multi-view data analysis provides an effective means to integrate the distinct information sources which are inherent to many applications. Data clustering in a multi-view setting specifically aims to identify the most appropriate grouping for a collection of entities, where those entities (or their relationships) can be described from multiple perspectives. Leveraging recent advances in multi-objective clustering, we propose a new evolutionary method to tackle this challenge. Designed around a flexible and unbiased solution representation, together with strategies based on the minimum spanning tree and neighborhood relations, our algorithm optimizes multiple objectives simultaneously to effectively explore the space of candidate trade-offs between the data views. Through a series of experiments, we investigate the suitability of our proposal in the context of a bioinformatics application, clustering of plausible protein structures, and a diverse set of synthetic problems. The specific case of two data views is considered in this paper. The evaluation with respect to a variety of reference approaches demonstrates the effectiveness of our method in discovering high-quality partitions in a multiview setting. Robustness against unreliable data sources and the ability to automatically determine the number of clusters, are additional advantages evidenced by the results obtained.

A great number of studies show that considering information from multiple views results in better performance than their single-view counterparts. However, many previous works aiming to improve classification performance fail to effectively tackle the inter-view correlation or the intra-class variability. Analysis dictionary learning (ADL) has theoretic significance and practical potential in classification tasks. Based on the ADL, this paper proposes a new method, namely, multi-view analysis dictionary learning (MvADL) for image classification. Specifically, multi-view analysis dictionaries are designed to reduce the intra-class variability in the transformed space in the multi-view scenario. Then, a marginalized classification term is incorporated to integrate the semantic information into the basic dictionary learning model. In the marginalized classification term, a marginalized target learning strategy is applied to improve the flexibility and discriminability of the whole model. Besides, an iteratively optimizing algorithm is designed to solve the proposed MvADL. Experiments on benchmark data sets demonstrate the superiority of our proposed method.

The problem of multi-view transformation is associated with transforming available source views of a given object into unknown target views. To solve this problem, a Mutual-Encoding InfoGenerative Adversarial Networks (MEIGANs)-based algorithm is proposed in this paper. A mutual-encoding representation learning network is proposed to obtain multi-view representations, i.e., it guarantees through encoders different views of the same object are mapped to the common representation, which carries enough information with respect to the object itself. An InfoGenerative Adversarial Networks-based transformation network is proposed to transform multi-views of the given object, which carries the representation information in the generative models and discriminative models, guaranteeing the synthetic transformed view matches the source view. The advantages of the MEIGAN are that it bypasses direct mappings among different views, and can solve the problem of missing views in training data and the problem of mapping between transformed views and source views. Finally, experiments on incomplete data to complete data restoration tasks on MNIST, CelebA, and multi-view angle transformation tasks on 3-D rendered chairs and multi-view clothing show the proposed algorithm yields satisfactory transformation results.

Eye movement data collection is very expensive and laborious. Moreover, there are usually missing values. Assuming that we are collecting eye movement data from a set of images viewed by different users, there is a possibility that we will not able to collect the data of every user from every imageone or more views may not be represented in the image. We assume that the relationships among the views can be learnt from the whole collection of views (or items). The task is then to reproduce the missing part of the incomplete items from the relationships derived from the complete items and the known part of these items. Using certain properties of tensor algebra, we showed that this problem can be formulated consistently as a regression type learning task. Furthermore, there is a maximum margin based optimisation framework in which this problem can be solved in a tractable way. This problem is similar to learning to predict where a person is looking in an image. Therefore, we proposed an algorithm called Tensor-based Multi-View Learning(TMVL) in this paper. Furthermore, we also proposed a technique for improving prediction by introducing a new feature set obtained from Kronecker decomposition of the image fused with users eye movement data. Using this new feature can improve prediction performance markedly. The proposed approach was proven to be more effective than two well-known saliency detection techniques.

With the increase popularity of multi-view data, multi-view learning has attracted vital attentions in pattern recognition as well as machine learning. Most of existing methods apply in traditional single view learning. However, these methods neglect the complementary information among the views. The aim of multi-view is to discover complementary information and enhance the single view learning result. Multi-view is capable of capture incomplete and different types of information from multiple sources. However, multi-views may contain redundant information. Many multi-view methods assume that multi-views are generated from various view-specific generation matrices. This paper proposes the multi-view collaborative representation classification (MVCRC) algorithm which contains the information of different views and the connection of view-to-view. Experimental results conducted on five practical databases are used to confirm the effectiveness of the proposed approach.

Hyperspectral image (HSI) Clustering is an unsupervised task, which segments pixels into different groups without using labeled samples. In this paper, spatial-spectral based multi-view low-rank sparse subspace clustering (SSMLC) algorithm is proposed. Due to significant number of spectra bands HSI contains much more information than a regular image. These spectral information can be considered as multiview. In this paper, the spectral partitioning is applied to generate spectral views which contain correlated bands. Morphological features of the original HSI are taken as another view which contains spatial features. Principal components construct another view, which eliminates the noise in the original dataset. After the multi-view dataset is formed, multi-view low-rank sparse subspace clustering is applied to segment HSI. Our experiments show that the performance of the proposed SSMLC is better than other that of clustering algorithms such as sparse subspace clustering and low-rank sparse subspace clustering.

The primary objective of sentence segmentation process is to determine the sentence boundaries of a stream of words output by the automatic speech recognizers. Statistical methods developed for sentence segmentation requires a significant amount of labeled data which is time-consuming, labor intensive and expensive. In this work, we propose new multi-view semi-supervised learning strategies for sentence boundary classification problem using lexical, prosodic, and morphological information. The aim is to find effective semi-supervised machine learning strategies when only small sets of sentence boundary labeled data are available. We primarily investigate two semi-supervised learning approaches, called self-training and co-training. Different example selection strategies were also used for co-training, namely, agreement, disagreement and self-combined. Furthermore, we propose three-view and committee-based algorithms incorporating with agreement, disagreement and self-combined strategies using three disjoint feature sets. We present comparative results of different learning strategies on the sentence segmentation task. The experimental results show that the sentence segmentation performance can be highly improved using multi-view learning strategies that we proposed since data sets can be represented by three redundantly sufficient and disjoint feature sets. We show that the proposed strategies substantially improve the average baseline F-measure of 67.66% to 75.15% and 64.84% to 66.32% when only a small set of manually labeled data is available for Turkish and English spoken languages, respectively. This material is based upon work supported by the Scientific and Technological Research Council of Turkey (TUBITAK) (Project Number: 107E182 and Project Number: 111E228), Isik University Scientific Research Projects Fund (Project Number: 09A301 and Project Number: 14A201), TUBITAK BIDEB and J. William Fulbright Post-Doctoral Research Fellowship, USA fundings at SRI-International, Speech Technology and Research (STAR) Lab., Menlo Park, CA, USA and International Computer Science Institute (ICSI) Speech Group, University of California at Berkeley, CA, USA. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the funding agencies. The authors thank Gokhan Tur, Dilek Hakkani- Tur, Benoit Favre, Sebastien Cuendet, Murat Saraclar, Siddika Parlak, Erinc Dikici, Izel D. Revidi, Cenk Demiroglu and Fatih Ozaydin and Bogazici University Signal and Image Processing (BUSIM) Group for many helpful discussions. The authors also thank the anonymous reviewers for their useful comments on an earlier version of this paper. Publisher's Version

Images of facial expressions are often captured from various views as a result of either head movements or variable camera position. Existing methods for multiview and/or view-invariant facial expression recognition typically perform classification of the observed expression using either classifiers learned separately for each view or a single classifier learned for all views. However, these approaches ignore the fact that different views of a facial expression are just different manifestations of the same facial expression. By accounting for this redundancy, we can design more effective classifiers for the target task. To this end, we propose a discriminative shared Gaussian process latent variable model (DS-GPLVM) for multiview and view-invariant classification of facial expressions from multiple views. In this model, we first learn a discriminative manifold shared by multiple views of a facial expression. Subsequently, we perform facial expression classification in the expression manifold. Finally, classification of an observed facial expression is carried out either in the view-invariant manner (using only a single view of the expression) or in the multiview manner (using multiple views of the expression). The proposed model can also be used to perform fusion of different facial features in a principled manner. We validate the proposed DS-GPLVM on both posed and spontaneously displayed facial expressions from three publicly available datasets (MultiPIE, labeled face parts in the wild, and static facial expressions in the wild). We show that this model outperforms the state-of-the-art methods for multiview and view-invariant facial expression classification, and several state-of-the-art methods for multiview learning and feature fusion.

This paper introduces a novel multi-view multi-learner (MVML) active learning method, in which the different views are generated by a genetic algorithm (GA). The GA-based view generation method attempts to construct diverse, sufficient, and independent views by considering both inter- and intra-view confidences. Hyperspectral data inherently owns high dimensionality, which makes it suitable for multi-view learning algorithms. Furthermore, by employing multiple learners at each view, a more accurate estimation of the underlying data distribution can be obtained. We also implemented a spectral-spatial graph-based semi-supervised learning (SSL) method as the classifier, which improved the performance of the classification task in comparison with supervised learning. The evaluation of the proposed method was based on three different benchmark hyperspectral data sets. The results were also compared with other state-of-the-art AL-SSL methods. The experimental results demonstrated the efficiency and statistically significant superiority of the proposed method. The GA-MVML AL method improved the classification performances by 16.68%, 18.37%, and 15.1% for different data sets after 40 iterations.

In this paper, three novel classification algorithms aiming at (semi-)supervised action classification are proposed. Inspired by the effectiveness of discriminant subspace learning techniques and the fast and efficient Extreme Learning Machine (ELM) algorithm for Single-hidden Layer Feedforward Neural networks training, the ELM algorithm is extended by incorporating discrimination criteria in its optimization process, in order to enhance its classification performance. The proposed Discriminant ELM algorithm is extended, by incorporating proper regularization in its optimization process, in order to exploit information appearing in both labeled and unlabeled action instances. An iterative optimization scheme is proposed in order to address multi-view action classification. The proposed classification algorithms are evaluated on three publicly available action recognition databases providing state-of-the-art performance in all the cases.

Several computational approaches have been proposed for inferring the affective state of the user, motivated for example by the goal of building improved interfaces that can adapt to the user@?s needs and internal state. While fairly good results have been obtained for inferring the user state under highly controlled conditions, a considerable amount of work remains to be done for learning high-quality estimates of subjective evaluations of the state in more natural conditions. In this work, we discuss how two recent machine learning concepts, multi-view learning and multi-task learning, can be adapted for user state recognition, and demonstrate them on two data collections of varying quality. Multi-view learning enables combining multiple measurement sensors in a justified way while automatically learning the importance of each sensor. Multi-task learning, in turn, tells how multiple learning tasks can be learned together to improve the accuracy. We demonstrate the use of two types of multi-task learning: learning both multiple state indicators and models for multiple users together. We also illustrate how the benefits of multi-task learning and multi-view learning can be effectively combined in a unified model by introducing a novel algorithm.

Multi-view learning studies how several views, different feature representations, of the same objects could be best utilized in learning. In other words, multi-view learning is analysis of co-occurrence data, where the observations are co-occurrences of samples in the views. Standard multi-view learning such as joint density modeling cannot be done in the absence of co-occurrence, when the views are observed separately and the identities of objects are not known. As a practical example, joint analysis of mRNA and protein concentrations requires mapping between genes and proteins. We introduce a data-driven approach for learning the correspondence of the observations in the different views, in order to enable joint analysis also in the absence of known co-occurrence. The method finds a matching that maximizes statistical dependency between the views, which is particularly suitable for multi-view methods such as canonical correlation analysis which has the same objective. We apply the method to translational metabolomics, to identify differences and commonalities in metabolic processes in different species or tissues. The metabolite identities and roles in the different species are not generally known, and it is necessary to search for a matching. In this paper we show, using different metabolomics measurement batches as the views so that the ground truth is known, that the metabolite identities can be reliably matched by a consensus of several matching solutions.

Protein X-ray crystallography images are usually studied for the purpose of identifying the three-dimensional structure of proteins at atomic resolution. These experiments produce hundreds and thousands of experimental results, leading to the need for automatic rather than manual analysis. The experiments are also time dependent and require monitoring over a long period of time, beginning from an hour of starting the experiment to a couple of months. The time gap between the imaged instances of these experiments also varies from frame to frame. This thesis introduces the use of difference images of an x-ray crystallography sequence in classifying the protein x-ray crystallography images, and exploits their various features along with the features of the original images. By further experiments using several combinations of difference images along with the original images, the significance of difference images in accurate classification of x-ray crystallography is established. At this stage, two independent feature sets for protein x-ray crystallography images are available. Based on these two feature sets, multi-view learning is employed for the purpose of classification of x-ray crystallography images, and random forests is shown to be a good base classifier for both views. During multi-view learning, different co-training methods are experimented with to determine the best possible classification result for protein x-ray crystallography images. After establishing by manual analysis that the same outcomes can be obtained via several different pathways in a crystallography time sequence, the results obtained from single frame classification experiments are used in the classification of protein x-ray crystallography sequences. Finally, the time varying nature of the x-ray crystallography experiments is further exploited together with the results of the previous experiments, to predict the final outcomes of crystallography experiments based on the first few frames. This provides the ability, for the first time, to determine whether to continue observing a sequence or to discard it as likely to be a non-crystallisation outcome.

Multivariate multi-way ANOVA-type models are the default tools for analyzing experimental data with multiple independent covariates. However, formulating standard multi-way models is not possible when the data comes from different sources or in cases where some covariates have (partly) unknown structure, such as time with unknown alignment. The “small n, large p”, large dimensionality p with small number of samples n, settings bring further problems to the standard multivariate methods. We extend our recent graphical multi-way model to three general setups, with timely applications in biomedicine: (i) multi-view learning with paired samples, (ii) one covariate is time with unknown alignment, and (iii) multi-view learning without paired samples.

In many scientific settings data can be naturally partitioned into variable groupings called views. Common examples include environmental (1st view) and genetic information (2nd view) in ecological applications, chemical (1st view) and biological (2nd view) data in drug discovery. Multi-view data also occur in text analysis and proteomics applications where one view consists of a graph with observations as the vertices and a weighted measure of pairwise similarity between observations as the edges. Further, in several of these applications the observations can be partitioned into two sets, one where the response is observed (labeled) and the other where the response is not (unlabeled). The problem for simultaneously addressing viewed data and incorporating unlabeled observations in training is referred to as multi-view transductive learning. In this work we introduce and study a comprehensive generalized fixed point additive modeling framework for multi-view transductive learning, where any view is represented by a linear smoother. The problem of view selection is discussed using a generalized Akaike Information Criterion, which provides an approach for testing the contribution of each view. An efficient implementation is provided for fitting these models with both backfitting and local-scoring type algorithms adjusted to semi-supervised graph-based learning. The proposed technique is assessed on both synthetic and real data sets and is shown to be competitive to state-of-the-art co-training and graph-based techniques., Comment: Published in at http://dx.doi.org/10.1214/08-AOAS202 the Annals of Applied Statistics (http://www.imstat.org/aoas/) by the Institute of Mathematical Statistics (http://www.imstat.org)

International audience; With the rapid advancements of sensor technologies and mobile computing, Mobile Crowd-Sensing (MCS) has emerged as a new paradigm to collect massive-scale rich trajectory data. Nomadic sensors empower people and objects with the capability of reporting and sharing observations on their state, their behavior and/or their surrounding environments. Processing and mining multi-source sensor data in MCS raise several challenges due to their multi-dimensional nature where the measured parameters (i.e., dimensions) may differ in terms of quality, variabilty, and time scale. We consider the context of air quality MCS, and focus on the task of mining the context from the MCS data. Relating the measures to their context is crucial to interpret them and analyse the participant's exposure. This paper investigates the feasibility of recognizing the human's context (called herein micro-environment) in an environmental MCS scenario. We put forward a multi-view learning approach, that we adapt to our context, and implement it along with other time series classification approaches. The experimental results, applied to real MCS data, not only confirm the power of MCS data in characterizing the micro-environment, but also show a moderate impact of the integration of mobility data in this recognition. Furthermore, multi-view learning shows similar performance as the reference deep learning algorithm, without requiring specific hardware.