Search

Most existing multi‐view stereo (MVS) methods fail to consider global context information in the stage of feature extraction and cost aggregation. As transformers have shown remarkable performance on various vision tasks due to their ability to perceive global contextual information, this paper proposes a transformer‐based feature enhancement network (TF‐MVSNet) to facilitate feature representation learning by combining local features (both 2D and 3D) with long‐range contextual information. To reduce memory consumption of feature matching, the cross‐attention mechanism is leveraged to efficiently construct 3D cost volumes under the epipolar constraint. Additionally, a colour‐guided network is designed to refine depth maps at a coarse stage, hence reducing incorrect depth predictions at a fine stage. Extensive experiments were performed on the DTU dataset and Tanks and Temples (T&T) benchmark and results are reported. [ABSTRACT FROM AUTHOR]

Deep network design is a fundamental challenge. A right trade‐off between depth and complexity of convolutional neural networks is of significant importance to applications in low‐level vision tasks. Wider feature maps could be beneficial to performance and generality but would increase computational complexity. In this paper, we rethink the balance between width of the feature maps and depth of the network especially for image restoration tasks including deblurring, dehazing, super‐resolution, and denoising. We explore a new approach to network structure by encouraging more depth to deal with restoration requirements while decreasing the width of some feature maps. Such a slimmer and deeper approach can enhance the performance while maintaining the same level of computational costs. We have experimentally evaluated the performances of the proposed approach on four image restoration tasks and obtained state‐of‐the‐art results on quantitative measures and qualitative assessments, demonstrating the effectiveness of the approach. [ABSTRACT FROM AUTHOR]

Facial expressions contain a great deal of information in inter-personal communication and social interaction and have a great impact on face recognition. This paper investigates expression synthesis methods based on deep learning and their performance in expression classification and face recognition. We propose a new synthesis method based on the active appearance model and CycleGAN. The eigendecomposition method is used to obtain the expression shape of a target subject through linear operations. An expression intensity coefficient is developed to control the dynamics of generated expressions. Subsequently, a generative model is introduced for the synthesis on textures. The identity loss is integrated into the traditional CycleGAN model, incorporated with the adversarial loss and the cycle consistency loss, to better preserve texture attributes and identity information of target subjects. Experiments and comparisons show that the proposed method offers marked improvements in verification of expressions and identities, as well as in generalization across databases. [ABSTRACT FROM AUTHOR]

Over the last decade, convolutional neural networks (CNNs) have allowed remarkable advances in single image super-resolution (SISR). In general, recovering high-frequency features is crucial for high-performance models. High-frequency features suffer more serious damages than low-frequency features during downscaling, making it hard to recover edges and textures. In this paper, we attempt to guide the network to focus more on high-frequency features in restoration from both channel and spatial perspectives. Specifically, we propose a high-frequency channel attention (HFCA) module and a frequency contrastive learning (FCL) loss to aid the process. For the channel-wise perspective, the HFCA module rescales channels by predicting statistical similarity metrics of the feature maps and their high-frequency components. For the spatial perspective, the FCL loss introduces contrastive learning to train a spatial mask that adaptively assigns high-frequency areas with large scaling factors. We incorporate the proposed HFCA module and FCL loss into an EDSR baseline model to construct the proposed lightweight high-frequency channel contrastive network (HFCCN). Extensive experimental results show that it can yield markedly improved or competitive performances compared to the state-of-the-art networks of similar model parameters. [ABSTRACT FROM AUTHOR]

The research paper focuses on a challenging task faced in blind image deblurring (BID). It relates to the estimation of arbitrarily shaped (nonparametric or complex shaped) point spread functions (PSFs) of motion blur caused by camera handshake. These PSFs exhibit much more complex shapes than their parametric counterparts and deblurring, in this case, requires intricate ways to estimate the blur and effectively remove it. This research work introduces a novel blind deblurring scheme visualized for deblurring images corrupted by arbitrarily shaped PSFs. It is based on genetic algorithm and utilizes the Blind/Reference-less Image Spatial QUality Evaluator (BRISQUE) measure as the fitness function for arbitrarily shaped PSF estimation. The proposed BID scheme has been compared with other state-of-the-art single image motion deblurring schemes as benchmarks. Validation has been carried out on the standard real-life blurred images. Results of both benchmark and real images are presented. For real-life blurred images, the proposed BID scheme using BRISQUE converges in close vicinity of the original blurring functions. However, the benchmark schemes fail to effectively restore the real blurred images. The proposed scheme surpasses on average of seven percent higher image quality as compared to the benchmark schemes. [ABSTRACT FROM AUTHOR]

Facial expression synthesis has drawn increasing attention in computer vision, graphics and animation. Recently, generative adversarial nets (GANs) have become a new perspective for face synthesis and have had remarkable success in generating photorealistic images and image‐to‐image translation. In this study, the authors present an appearance‐based facial expression synthesis framework, ApprGAN, by combining shape and texture and introducing cycle consistency and identity mapping into the adversarial learning. Specifically, given an input face image, a pair of shape and texture generators are trained for synthetic shape deformation and expression detail generation, respectively. Extensive experiments on expression synthesis and cross‐database synthesis were conducted, together with comparisons with the existing methods. Results of expression synthesis and quantitative verification on various databases show the effectiveness of ApprGAN in synthesising photorealistic and identity‐preserving expressions and its marked improvement over the existing methods. [ABSTRACT FROM AUTHOR]

Over the last decades, expression classification and face recognition have received substantial attention in computer vision and pattern recognition with more recent efforts focusing on understanding and modelling expression variations. In this paper, we present an expression classification and expression-invariant face recognition method by synthesising photorealistic expression manifolds to expand the gallery set. By means of synthesising expression images from neutral faces, more within-subject variability can be obtained. Eigentransformation is utilised to generate both shape and expression details for novel subjects. Expression classification and face recognition are then performed on the extended training set with synthesised expressions. Experimental results on various datasets show that the proposed method is robust for recognising various expressions and faces with varying degrees of expression. Comprehensive experiments conducted and comparisons with the existing methods are reported. Cross-database synthesis and effect of landmark quality are also studied. [ABSTRACT FROM AUTHOR]

Canonical correlation analysis (CCA) is a useful dimensionality reduction method and has been widely used in projecting multi-view data. However, CCA learns from training data consisting of only target classes, ignoring Universum data that belongs to none of the target classes but comes from the same domain as the target classes. Recently, incorporating Universum data in learning has been used to gain more prior knowledge about the application domain and has been shown to achieve favorable improvements. In this paper, we extend CCA with Universum learning for multi-view data and the proposed method is termed as Universum CCA (UCCA). Due to the fact that Universum data in each view does not belong to any target class, correlation between Universum data and target data should be minimized. Consequently, UCCA aims to find basis vectors in multiple views to ensure that correlations between projections of target data are mutually maximized but correlations between projections of Universum data and target data mutually minimized. UCCA can be expressed as a generalized eigenvalue problem and the extracted features express patterns more distinctly. The experimental results on several real-world datasets demonstrate its marked improvements over conventional methods. [ABSTRACT FROM AUTHOR]

This paper presents a compact and efficient yet powerful binary framework based on image gradients for robust facial representation. It is termed as Binary Gradient Patterns (BGP). To discover underlying local structures in the gradient domain, image gradients are computed from multiple directions and encoded into a set of binary strings. Certain types of these binary strings have meaningful local structures and textures, as they detect micro oriented edges and retain strong local orientation, thus enabling great discrimination. Face representations by these structural BGP histograms exhibit profound robustness against various facial image variations, in particular illumination. The binary strategy realized by local correlations substantially simplifies the computational complexity and achieves extremely efficient processing with only 0.0032 s in Matlab for a typical image. Furthermore, the discrimination power of the BGP has been enhanced on a set of orientations of the image-gradient magnitudes. Extensive experimental results on various benchmarks demonstrate that the BGP-based representations significantly improve over the existing local descriptors and state-of-the-art methods in the terms of discrimination, robustness and complexity and in many cases the improvements are substantial. Combining with the deep networks, the proposed descriptors can further improve the performance of the deep networks on real-world datasets. Matlab codes for the BGP-based descriptors are available at: http://www.eee.manchester.ac.uk/our-research/research-groups/sisp/research-areas/vip/software/ . [ABSTRACT FROM AUTHOR]

Abstract: Nowadays, there exist various methods for modelling and forecasting foreign exchange (FX) rates including economical models, statistical methods and learning neural networks. Dealing with the problems of nonstationarity and nonlinearity has been a challenge. In this paper, we propose a combined neural model for effectively tackling the problems. The model is termed as neural gas mixture of autoregressive (NGMAR) models and it organizes the mixture of autoregressive models in the way of the neural gas. By taking the advantages of dynamic neighbourhood rankings of neural gas and the more appropriate similarity measure of the sum of autocorrelation coefficients, the model is able to effectively model and forecast nonstationary and nonlinear time series. The NGMAR has been tested on several benchmark data sets as well as a variety of FX rates. The experimental results show that the proposed method outperforms significantly other methods, in terms of normalized root mean squared error and correct trend prediction percentage. [Copyright &y& Elsevier]

The principle of Independent Component Analysis (ICA) has been used in blind signal separation and deconvolution problems. In image restoration, such methods are often computationally intensive and ringing and noise amplification artifacts from the deblurring process greatly affect the image statistics and vary the calculated non-Gaussianity measures. To overcome the problems, we propose an enhanced scheme that employs the non-Gaussianity principle of ICA on the spectrum rather than the image data itself. That is, the spectral kurtosis is used as a measure of non-Gaussianity during the deblurring process. The deblurring process measures the non-Gaussianity of the image spectrum of the estimated images and the value maximizes at the true blurring kernel. The optimal solution is sought through a Genetic Algorithm. The scheme is simple and efficient and does not require any prior knowledge about the image or the blurring process. Validations have been carried out on various examples and they show that spectral non-Gaussianity optimizes on the parameters in a close vicinity of the original blurring functions. Results are presented for both benchmark and real images. The proposed method achieves marked improved results over the existing methods. [ABSTRACT FROM AUTHOR]

Abstract: The curse of dimensionality has prompted intensive research in effective methods of mapping high dimensional data. Dimensionality reduction and subspace learning have been studied extensively and widely applied to feature extraction and pattern representation in image and vision applications. Although PCA has long been regarded as a simple, efficient linear subspace technique, many nonlinear methods such as kernel PCA, local linear embedding, and self-organizing networks have been proposed recently for dealing with increasingly complex nonlinear data. The intensive research in nonlinear methods often creates an impression that they are highly superior and preferred, though often limited experiments were given and the results not tested on significance. In this paper, we systematically investigate and compare the capabilities of various linear and nonlinear subspace methods for face representation and recognition. The performances of these methods are analyzed and discussed along with statistical significance tests on obtained results. The experiments on a range of data sets show that nonlinear methods do not always outperform linear ones, especially on data sets containing noise and outliers or having discontinuous or multiple submanifolds. Certain nonlinear methods with certain classifiers do yield better performances consistently than others. However, the differences among them are small and in most cases are not significant. A measure is used to quantify the nonlinearity of a data set in a subspace. It explains that good performances are achievable in reduced dimensions of low degree of nonlinearity. [Copyright &y& Elsevier]

The empirical mode decomposition (EMD) method can adaptively decompose a non-stationary time series into a number of amplitude or frequency modulated functions known as intrinsic mode functions. This paper combines the EMD method with information analysis and presents a framework of information-preserving EMD. The enhanced EMD method has been exploited in the analysis of neural recordings. It decomposes a signal and extracts only the most informative oscillations contained in the non-stationary signal. Information analysis has shown that the extracted components retain the information content of the signal. More importantly, a limited number of components reveal the main oscillations presented in the signal and their instantaneous frequencies, which are not often obvious from the original signal. This information-coupled EMD method has been tested on several field potential datasets for the analysis of stimulus coding in visual cortex, from single and multiple channels, and for finding information connectivity among channels. The results demonstrate the usefulness of the method in extracting relevant responses from the recorded signals. An investigation is also conducted on utilizing the Hilbert phase for cases where phase information can further improve information analysis and stimulus discrimination. The components of the proposed method have been integrated into a toolbox and the initial implementation is also described. [ABSTRACT FROM AUTHOR]

Abstract: Nowadays a great deal of effort has been made in order to gain advantages in foreign exchange (FX) rates predictions. However, most existing techniques seldom excel the simple random walk model in practical applications. This paper describes a self-organising network formed on the basis of a mixture of adaptive autoregressive models. The proposed network, termed self-organising mixture autoregressive (SOMAR) model, can be used to describe and model nonstationary, nonlinear time series by means of a number of underlying local regressive models. An autocorrelation coefficient-based measure is proposed as the similarity measure for assigning input samples to the underlying local models. Experiments on both benchmark time series and several FX rates have been conducted. The results show that the proposed method consistently outperforms other local time series modelling techniques on a range of performance measures including the mean-square-error, correct trend predication percentage, accumulated profit and model variance. [Copyright &y& Elsevier]

Abstract: This paper describes a hybrid model formed by a mixture of various regressive neural network models, such as temporal self-organising maps and support vector regressions, for modelling and prediction of foreign exchange rate time series. A selected set of influential trading indicators, including the moving average convergence/divergence and relative strength index, are also utilised in the proposed method. A genetic algorithm is applied to fuse all the information from the mixture regression models and the economical indicators. Experimental results and comparisons show that the proposed method outperforms the global modelling techniques such as generalised autoregressive conditional heteroscedasticity in terms of profit returns. A virtual trading system is built to examine the performance of the methods under study. [Copyright &y& Elsevier]

Modelling non-stationary time series has been a difficult task for both parametric and nonparametric methods. One promising solution is to combine the flexibility of nonparametric models with the simplicity of parametric models. In this paper, the self-organising mixture autoregressive (SOMAR) network is adopted as a such mixture model. It breaks time series into underlying segments and at the same time fits local linear regressive models to the clusters of segments. In such a way, a global non-stationary time series is represented by a dynamic set of local linear regressive models. Neural gas is used for a more flexible structure of the mixture model. Furthermore, a new similarity measure has been introduced in the self-organising network to better quantify the similarity of time series segments. The network can be used naturally in modelling and forecasting non-stationary time series. Experiments on artificial, benchmark time series (e.g. Mackey-Glass) and real-world data (e.g. numbers of sunspots and Forex rates) are presented and the results show that the proposed SOMAR network is effective and superior to other similar approaches. [ABSTRACT FROM AUTHOR]

For reality applications of semantic segmentation, inference speed and memory usage are two important factors. To address these challenges, we propose a lightweight feature pyramid encoding network (FPENet) for semantic segmentation with a good trade-off between accuracy and speed. We use a series of feature pyramid encoding (FPE) blocks to encode context at multiple scales in the encoder. Each FPE block consists of different depthwise dilated convolutions that perform as a spatial pyramid to extract features and reduce computational costs. During training, a one-shot neural architecture search algorithm is adopted to find the optimal structure for each FPE block from a large search space with a small search cost. After the search for the encoder, a mutual embedding upsample module is introduced in the decoder, consisting of two attention blocks. The encoder-decoder attention mechanism is used to help aggregate efficiently high-level semantic features and low-level spatial details. The proposed network outperforms the existing real-time methods with fewer parameters and improved inference speed on the Cityscapes and CamVid benchmark datasets. Specifically, it achieved 72.3% mean IoU on the Cityscapes test set with only 0.4 M parameters and 192.6 FPS speed on an Nvidia Titan V100 GPU, and 73.4% mean IoU with 116.2 FPS when running on higher resolution images. [ABSTRACT FROM AUTHOR]

Abstract: The self-organising map (SOM) and its variant, visualisation induced SOM (ViSOM), have been known to yield similar results to multidimensional scaling (MDS). However, the exact connection has not been established. In this paper, a review on the SOM and its cost function and topological measures is provided first. We then examine the exact scaling effect of the SOM and ViSOM from their objective functions. The SOM is shown to produce a qualitative, nonmetric scaling, while the local distance-preserving ViSOM produces a quantitative or metric scaling. Their relationship with the principal manifold is also discussed. The SOM-based methods not only produce topological or metric scaling but also provide a principal manifold. Furthermore a growing ViSOM is proposed to aid the adaptive embedding of highly nonlinear manifolds. Examples and comparisons with other embedding methods such as Isomap and local linear embedding are also presented. [Copyright &y& Elsevier]

Abstract: The kernel method has become a useful trick and has been widely applied to various learning models to extend their nonlinear approximation and classification capabilities. Such extensions have also recently occurred to the Self-Organising Map (SOM). In this paper, two recently proposed kernel SOMs are reviewed, together with their link to an energy function. The Self-Organising Mixture Network is an extension of the SOM for mixture density modelling. This paper shows that with an isotropic, density-type kernel function, the kernel SOM is equivalent to a homoscedastic Self-Organising Mixture Network, an entropy-based density estimator. This revelation on the one hand explains that kernelising SOM can improve classification performance by acquiring better probability models of the data; but on the other hand it also explains that the SOM already naturally approximates the kernel method. [Copyright &y& Elsevier]

Abstract: When browsing a large set of unstructured documents, it is advantageous if the documents have been organised and presented in a way that makes navigation efficient, understanding underlying concepts easy and locating related information quickly. This paper proposes a new method termed Treeview self-organising maps (Treeview SOMs) for clustering and organising text documents by means of a series of independently and automatically created, hierarchical one-dimensional SOMs. The method generates a topological taxonomy tree for a set of unstructured text documents in terms of presentation and visualisation. The documents are organised in a hierarchy of dynamically generated and automatically validated topics extracted from the corpus of the documents. The results presented in a labelled tree view, clearly show underlying contents of the documents and can help browsing the document set more efficiently than those of previous work using SOMs or hierarchical clustering methods. A brief overview on general document clustering and a review on SOM-based document analysis methods are also provided together with a comparison among them. [Copyright &y& Elsevier]

Abstract: The self-organising map (SOM) is finding more and more applications in a wide range of fields, such as clustering, pattern recognition and visualisation. It has also been employed in knowledge management and information retrieval. We propose an alternative to existing 2-dimensional SOM based methods for document analysis. The method, termed Adaptive Topological Tree Structure (ATTS), generates a taxonomy of underlying topics from a set of unclassified, unstructured documents. The ATTS consists of a hierarchy of adaptive self-organising chains, each of which is validated independently using a proposed entropy-based Bayesian information criterion. A node meeting the expansion criterion spans a child chain, with reduced vocabulary and increased specialisation. The ATTS creates a topological tree of topics, which can be browsed like a content hierarchy and reflects the connections between related topics at each level. A review is also given on the existing neural network based methods for document clustering and organisation. Experimental results on real-world datasets using the proposed ATTS method are presented and compared with other approaches. The results demonstrate the advantages of the proposed validation criteria and the efficiency of the ATTS approach for document organisation, visualisation and search. It shows that the proposed methods not only improve the clustering results but also boost the retrieval. [Copyright &y& Elsevier]

This paper proposes the use of self-organizing maps (SOMs) to the blind source separation (BSS) problem for nonlinearly mixed signals corrupted with multiplicative noise. After an overview of some signal denoising approaches, we introduce the generic independent component analysis (ICA) framework, followed by a survey of existing neural solutions on ICA and nonlinear ICA (NLICA). We then detail a BSS method based on SOMs and intended for image denoising applications. Considering that the pixel intensities of raw images represent a useful signal corrupted with noise, we show that an NLICA-based approach can provide a satisfactory solution to the nonlinear BSS (NLBSS) problem. Furthermore, a comparison between the standard SOM and a modified version, more suitable for dealing with multiplicative noise, is made. Separation results obtained from test and real images demonstrate the feasibility of our approach. [Copyright &y& Elsevier]

The self-organising map (SOM) has been successfully employed as a nonparametric method for dimensionality reduction and data visualisation. However, for visualisation the SOM requires a colouring scheme to imprint the distances between neurons so that the clustering and boundaries can be seen. Even though the distributions of the data and structures of the clusters are not faithfully portrayed on the map. Recently an extended SOM, called the visualisation-induced SOM (ViSOM) has been proposed to directly preserve the distance information on the map, along with the topology. The ViSOM constrains the lateral contraction forces between neurons and hence regularises the inter-neuron distances so that distances between neurons in the data space are in proportion to those in the map space. This paper shows that it produces a smooth and graded mesh in the data space and captures the nonlinear manifold of the data. The relationships between the ViSOM and the principal curve/surface are analysed. The ViSOM represents a discrete principal curve or surface and is a natural algorithm for obtaining principal curves/surfaces. Guidelines for applying the ViSOM constraint and setting the resolution parameter are also provided, together with experimental results and comparisons with the SOM, Sammon mapping and principal curve methods. [Copyright &y& Elsevier]

Presents information on a study which considered the design of multiuser receivers for multicarrier direct-sequence code-division multiple access communication systems over fading channels. Problem formulation; Derivation of a class of spreading codes to close the performance gap between the despreading and combining and minimum mean-squared error (MMSE) receivers; Code design criterion that minimizes the average MSE given the channel statistics; Conclusions.

In this paper, we present an extensive review on deep learning methods for image restoration tasks. Deep learning techniques, led by convolutional neural networks, have received a great deal of attention in almost all areas of image processing, especially in image classification. However, image restoration is a fundamental and challenging topic and plays significant roles in image processing, understanding and representation. It typically addresses image deblurring, denoising, dehazing and super-resolution. There are substantial differences in the approaches and mechanisms in deep learning methods for image restoration. Discriminative learning based methods are able to deal with issues of learning a restoration mapping function effectively, while optimisation models based methods can further enhance the performance with certain learning constraints. In this paper, we offer a comparative study of deep learning techniques in image denoising, deblurring, dehazing, and super-resolution, and summarise the principles involved in these tasks from various supervised deep network architectures, residual or skip connection and receptive field to unsupervised autoencoder mechanisms. Image quality criteria are also reviewed and their roles in image restoration are assessed. Based on our analysis, we further present an efficient network for deblurring and a couple of multi-objective training functions for super-resolution restoration tasks. The proposed methods are compared extensively with the state-of-the-art methods with both quantitative and qualitative analyses. Finally, we point out potential challenges and directions for future research. [ABSTRACT FROM AUTHOR]

Classical active flow control (AFC) methods based on solving the Navier–Stokes equations are laborious and computationally intensive even with the use of reduced-order models. Data-driven methods offer a promising alternative for AFC, and they have been applied successfully to reduce the drag of two-dimensional bluff bodies, such as a circular cylinder, using deep reinforcement-learning (DRL) paradigms. However, due to the onset of weak turbulence in the wake, the standard DRL method tends to result in large fluctuations in the unsteady forces acting on the cylinder as the Reynolds number increases. In this study, a Markov decision process (MDP) with time delays is introduced to model and quantify the action delays in the environment in a DRL process due to the time difference between control actuation and flow response along with the use of a first-order autoregressive policy (ARP). This hybrid DRL method is applied to control the vortex-shedding process from a two-dimensional circular cylinder using four synthetic jet actuators at a freestream Reynolds number of 400. This method has yielded a stable and coherent control, which results in a steadier and more elongated vortex formation zone behind the cylinder, hence, a much weaker vortex-shedding process and less fluctuating lift and drag forces. Compared to the standard DRL method, this method utilizes the historical samples without additional sampling in training, and it is capable of reducing the magnitude of drag and lift fluctuations by approximately 90% while achieving a similar level of drag reduction in the deterministic control at the same actuation frequency. This study demonstrates the necessity of including a physics-informed delay and regressive nature in the MDP and the benefits of introducing ARPs to achieve a robust and temporal-coherent control of unsteady forces in active flow control. [ABSTRACT FROM AUTHOR]

Satellite images are an extremely valuable resource in the aftermath of natural disasters such as hurricanes and tsunamis where they can be used for risk assessment and disaster management. In order to provide timely and actionable information for disaster response, a framework utilising segmentation neural networks is proposed to identify impacted areas and accessible roads in post-disaster scenarios. The effectiveness of pretraining with ImageNet -for the task of aerial image segmentation has been analysed and performances of popular segmentation models compared. Experimental results show that pretraining on ImageNet usually improves the segmentation performance for a number of models. Open data available from OpenStreetMap (OSM) is used for training, forgoing the need for time-consuming manual annotation. The method also makes use of graph theory to update road network data available from OSM and to detect the changes caused by a natural disaster. Extensive experiments on data from the 2018 tsunami that struck Palu, Indonesia show the effectiveness of the proposed framework. ENetSeparable, with 30% fewer parameters compared to ENet, achieved comparable segmentation results to that of the state-of-the-art networks. [ABSTRACT FROM AUTHOR]

In image classification, deriving robust image representations is a key process that determines the performance of vision systems. Numerous image features and descriptors have been developed manually over the years. As an alternative, however, deep neural networks, in particular convolutional neural networks (CNNs), have become popular for learning image features or representations from data and have demonstrated remarkable performance in many real-world applications. But CNNs often require huge amount of labelled data, which may be prohibitive in many applications, as well as long training times. This paper considers an alternative, data-independent means of obtaining features for CNNs. The proposed framework makes use of the Markov random field (MRF) and self-organising map (SOM) to generate basic features and model both intra- and inter-image dependencies. Various MRF textures are synthesized first, and are then clustered by a convolutional translation-invariant SOM, to form generic image features. These features can be directly applied as early convolutional filters of the CNN, leading to a new way of deriving effective features for image classification. The MRF framework also offers a theoretical and transparent way to examine and determine the influence of image features on performance of CNNs. Comprehensive experiments on the MNIST, rotated MNIST, CIFAR-10 and CIFAR-100 datasets were conducted with results outperforming most state-of-the-art models of similar complexity. [ABSTRACT FROM AUTHOR]

In this paper, a novel zonal machine learning (ML) approach for Reynolds-averaged Navier–Stokes (RANS) turbulence modeling based on the divide-and-conquer technique is introduced. This approach involves partitioning the flow domain into regions of flow physics called zones, training one ML model in each zone, then validating and testing them on their respective zones. The approach was demonstrated with the tensor basis neural network (TBNN) and another neural net called the turbulent kinetic energy neural network (TKENN). These were used to predict Reynolds stress anisotropy and turbulent kinetic energy, respectively, in test cases of flow over a solid block, which contain regions of different flow physics including separated flows. The results show that the combined predictions given by the zonal TBNNs and TKENNs were significantly more accurate than their corresponding standard non-zonal models. Most notably, shear anisotropy component in the test cases was predicted at least 20% and 55% more accurately on average by the zonal TBNNs compared to the non-zonal TBNN and RANS, respectively. The Reynolds stress constructed with the zonal predictions was also found to be at least 23% more accurate than those obtained with the non-zonal approach and 30% more accurate than the Reynolds stress predicted by RANS on average. These improvements were attributed to the shape of the zones enabling the zonal models to become highly locally optimized at predicting the output. [ABSTRACT FROM AUTHOR]

No abstract received. [ABSTRACT FROM AUTHOR]

Given a photo of a subject, ability to generate a caricature image that captures distinct characteristics of the subject but with certain exaggeration of their prominent features is of fundamental importance to image processing and facial recognition. There are two main challenges in this task: shape exaggeration and style transfer. The former morphs and exaggerates key facial features of the subject, while the latter generates caricature images in a certain artistic style. In this paper, we propose a CAricature Style Transfer (CAST) framework for caricature generation. There are two modules in the proposed framework. The first is a geometric warping module. Different from the existing style transfer methods, we incorporate the Whitening and Coloring Transformation (WCT) in the geometric style transfer. The WCT is learned on photo and caricature landmarks or the caricature landmark space of a specific artist and is capable of transforming input photo landmarks to caricature landmarks. The second module is a texture style rendering module. We propose a new style transfer method by considering a semantic region-aligned style transfer via affinity constraint. Given a reference caricature image as the style reference, this module is capable of transferring styles between the same or similar semantic regions in caricatures and photos. Furthermore, it can transfer visual attributes of the reference caricatures (such as mouth shape and expressions) to the output caricatures. Experiments have shown desirable effects of the proposed method in transferring both the geometric and artistic texture styles of caricatures. Both qualitative and quantitative results show that the CAST framework is more effective compared than the state-of-the-art caricature generation methods. [ABSTRACT FROM AUTHOR]

Cassava brown streak disease (CBSD) is an emerging viral disease that can greatly reduce cassava productivity, while causing only mild aerial symptoms that develop late in infection. Early detection of CBSD enables better crop management and intervention. Current techniques require laboratory equipment and are labour intensive and often inaccurate. We have developed a handheld active multispectral imaging (A-MSI) device combined with machine learning for early detection of CBSD in real-time. The principal benefits of A-MSI over passive MSI and conventional camera systems are improved spectral signal-to-noise ratio and temporal repeatability. Information fusion techniques further combine spectral and spatial information to reliably identify features that distinguish healthy cassava from plants with CBSD as early as 28 days post inoculation on a susceptible and a tolerant cultivar. Application of the device has the potential to increase farmers' access to healthy planting materials and reduce losses due to CBSD in Africa. It can also be adapted for sensing other biotic and abiotic stresses in real-world situations where plants are exposed to multiple pest, pathogen and environmental stresses. [ABSTRACT FROM AUTHOR]

Massively complex systems, such as social networks (Camacho, Panizo-LLedot, Bello-Orgaz, Gonzalez-Pardo, & Cambria, 2020; Lara-Cabrera et al., 2017), renewable energy problems (Twidell & Weir, 2015), or Internet-of-Things problems (Lin et al., 2017), generate massive amounts of data. This special issue welcomed submissions of original papers introducing research results on all the aspects covering the application of CI algorithms to massively complex systems, ranging from concepts and theoretical developments to advanced technologies and innovative applications. GRASP algorithm is also used in the paper by Pérez-Peló, Sánchez-Oro, & Duarte (2020), where the authors tackle the I i -separator problem. [Extracted from the article]

This paper presents a novel model for performing classification and visualization of high-dimensional data by means of combining two enhancing techniques. The first is a semi-supervised learning, an extension of the supervised learning used to incorporate unlabeled information to the learning process. The second is an ensemble learning to replicate the analysis performed, followed by a fusion mechanism that yields as a combined result of previously performed analysis in order to improve the result of a single model. The proposed learning schema, termed S2-Ensemble, is applied to several unsupervised learning algorithms within the family of topology maps, such as the Self-Organizing Maps and the Neural Gas. This study also includes a thorough research of the characteristics of these novel schemes, by means quality measures, which allow a complete analysis of the resultant classifiers from the viewpoint of various perspectives over the different ways that these classifiers are used. The study conducts empirical evaluations and comparisons on various real-world datasets from the UCI repository, which exhibit different characteristics, so to enable an extensive selection of situations where the presented new algorithms can be applied. [ABSTRACT FROM AUTHOR]

Eddy current testing (ECT) is an effective technique for evaluating depth of metal surface defects. However, in practice, evaluation primarily relies on the experience of an operator and is often carried out by manual inspection. In this article, we address the challenges of automatic depth evaluation of metal surface defects by virtual of state-of-the-art deep learning (DL) techniques. The main contributions are threefold. First, a highly integrated portable ECT device is developed, taking the advantage of an advanced field-programmable gate array (Zynq-7020 system on chip) and provides fast data acquisition and in-phase/quadrature demodulation. Second, a dataset, termed metal defects of different depths by ECT (MDDECT), is constructed using the ECT device by human operators and made openly available. It contains 48000 scans from 18 defects of different depths and liftoffs. Third, the depth evaluation problem is formulated as a time series classification problem, and various state-of-the-art 1-D residual convolutional neural networks are trained and evaluated on the MDDECT dataset. A 38-layer 1-D ResNeXt achieves an accuracy of 93.58% in discriminating the surface defects in a stainless steel sheet with depths from 0.3 to 2.0 mm in the resolution of 0.1 mm. In addition, the results show that the trained ResNeXt1D-38 model is immune to liftoff signals. [ABSTRACT FROM AUTHOR]

Surrogate models have been used as a substitute for traditional engineering simulation to get target simulation results faster. These surrogate models usually need sufficient data to fulfill their optimum performance. However, the acquisition of high quality datasets for cases like welding residual stress is both difficult and costly. This work developed a workflow that can rapidly develop machine learning based surrogate models to predict post-weld residual stresses using finite element (FE) simulation data. The workflow comprises five steps: (i) baseline FE model creation, (ii) FE model processing (iii) scripting (iv) data generation and (v) surrogate model development. The workflow presented in the paper automates all the steps involved in generating, calculating, and extracting results from FE simulations, with the aim of reducing the time and cost required for data acquisition. In addition, to minimize the possibility of FE analyses not completing successfully during the automated data generation process and to ensure that the training data can be obtained without any manual intervention, the FE model was simplified without significant loss of accuracy. A single weld bead-on-plate structure, taken from the NeT project Task Group 1, was used as a case study to validate the developed workflow. The overall aim of this work was to validate the feasibility of developing a surrogate model for specific welding conditions with no experimental dataset and with the aid of simulations, and to investigate the predictive performance of an Artificial Neural Network based surrogate model. Four welding parameters, arc advance speed, net energy input rate, arc travel length and position, were considered as the input for the surrogate model to predict the longitudinal stress along the line running across the sample thickness in the mid-length of the specimen. The relative root mean square error of the predictive surrogate model for the test dataset was 0.0024. Residual stress prediction based on the NeT-TG1 benchmark welding case met the R6 validation criteria and the developed model generates predictions that are in line with other methods used by contributors to the NeT-TG1 round robin. [ABSTRACT FROM AUTHOR]

Canonical correlation analysis (CCA) is a typical and useful learning paradigm in big data analysis for capturing correlation across multiple views of the same objects. When dealing with data with additional ordinal information, traditional CCA suffers from poor performance due to ignoring the ordinal relationships within the data. Such data is becoming increasingly common, as either temporal or sequential information is often associated with the data collection process. To incorporate the ordinal information into the objective function of CCA, the so-called ordinal discriminative CCA has been presented in the literature. Although ordinal discriminative CCA can yield better ordinal regression results, its performance deteriorates when data is corrupted with noise and outliers, as it tends to smear the order information contained in class centers. To address this issue, in this article we construct a robust manifold-preserved ordinal discriminative correlation regression (rmODCR). The robustness is achieved by replacing the traditional (l2-norm) class centers with lp-norm centers, where p is efficiently estimated according to the moments of the data distributions, as well as by incorporating the manifold distribution information of the data in the objective optimization. In addition, we further extend the robust manifold-preserved ordinal discriminative correlation regression to deep convolutional architectures. Extensive experimental evaluations have demonstrated the superiority of the proposed methods. [ABSTRACT FROM AUTHOR]

LiDAR provides highly accurate 3-D point clouds. However, data need to be manually labeled in order to provide subsequent useful information. Manual annotation of such data is time-consuming, tedious, and error prone, and hence, in this article, we present three automatic methods for annotating trees in LiDAR data. The first method requires high-density point clouds and uses certain LiDAR data attributes for the purpose of tree identification, achieving almost 90% accuracy. The second method uses a voxel-based 3-D convolutional neural network on low-density LiDAR data sets and is able to identify most large trees accurately but struggles with smaller ones due to the voxelization process. The third method is a scaled version of the PointNet++ method and works directly on outdoor point clouds and achieves an $F_{\mathrm{ score}}$ of 82.1% on the ISPRS benchmark data set, comparable to the state-of-the-art methods but with increased efficiency. [ABSTRACT FROM AUTHOR]

• A valid approach for crop/weed segmentation is presented. • An input is proposed to train the U-Net model with the ResNet-50 structure. • A CRF increases the segmentation accuracy by reducing incorrectly labelled pixels. • High mean IoU was obtained using the proposed method with 512x512 image size. Weeds are one of the most detrimental challenges to agriculture causing significant losses to beneficial yield potentials as they compete with crops for water, nutrients and sunlight. Early detection of weeds in the field is highly critical for taking appropriate actions, such as applying herbicides, mechanical removal or other remedial treatments, which are less effective or more resource intensive at later stages of crop growth. In this work, a deep learning method has been developed for weed detection. A sunflower dataset comprising multispectral images, the visible band (400–700 nm wavelengths in RGB) and Near Infrared (700–1000 nm wavelengths, NIR), captured at various days and times were used for the study. The deep learning model, 'U-Net', was trained with images from the cotyledon emergence through to the subsequent growth stages and tested on images of crops in the last stage of growth, where chemical treatments can be applied. The results of the U-Net were further enhanced by employing conditional random fields to achieve improved segmentation in terms of Intersection over Union (IoU). The proposed method using the Green (530–600 nm wavelengths) + Filtered-NIR + Normalised Difference Vegetation Index (NDVI) (Weier and Herring, Aug. 2000) channels as the input achieved the best mean IoU score of 0.883 on images of 512 × 512 pixels. In the same experiment, soil, crop and weed pixels were correctly predicted with 0.990 IoU, 0.906 IoU, and 0.753 IoU scores, respectively. The results show that the chosen input and the proposed methodology offer a viable approach for early-stage weed detection. [ABSTRACT FROM AUTHOR]

Ordinal regression (OR) is an important research topic in machine learning and has attracted extensive attention due to its wide applications. So far, a variety of methods have been proposed to perform OR, in which the class-center-induced threshold methods (like KDLOR and MOR) have received more attention, for their simplicity and promising performance. The class-center-induced ORs typically calculate the ordinal thresholds with class centers, which are typically derived from the l 2 -norm. Unfortunately, in such a way, the class means may be biased when the data is corrupted with outliers (i.e., non-i.i.d. noises) such that the resulting OR accuracy will be deteriorated. Motivated by the success of l p -norm in applications against noises, in this paper we propose a novel type of class centroid derived from the l p -norm (coined as l p -centroid) to overcome the drawbacks above, and provide an optimization algorithm and corresponding convergence analysis for computing the l p -centroid. To evaluate the effectiveness of l p -centroid in OR context against noises, we then combine the l p -centroid with two representative class-center-induced ORs, namely discriminant learning based and manifold learning based ORs. Finally, extensive OR experiments on synthetic and real-world datasets demonstrate the effectiveness and superiority of the proposed methods to related existing methods. [ABSTRACT FROM AUTHOR]

Novelty detection, or one-class classification, is of particular use in the analysis of high-integrity systems, in which examples of failure are rare in comparison with the number of examples of stable behaviour, such that a conventional multi-class classification approach cannot be taken. Support Vector Machines (SVMs) are a popular means of performing novelty detection, and it is conventional practice to use a train-validate-test approach, often involving cross-validation, to train the one-class SVM, and then select appropriate values for its parameters. An alternative method, used with multi-class SVMs, is to calibrate the SVM output into conditional class probabilities. A probabilistic approach offers many advantages over the conventional method, including the facility to select automatically a probabilistic novelty threshold. The contributions of this paper are (i) the development of a probabilistic calibration technique for one-class SVMs, such that on-line novelty detection may be performed in a probabilistic manner; and (ii) the demonstration of the advantages of the proposed method (in comparison to the conventional one-class SVM methodology) using case studies, in which one-class probabilistic SVMs are used to perform condition monitoring of a high-integrity industrial combustion plant, and in detecting deterioration in patient physiological condition during patient vital-sign monitoring. [ABSTRACT FROM AUTHOR]

Background: The use of spectral imaging within the plant phenotyping and breeding community has been increasing due its utility as a non-invasive diagnostic tool. However, there is a lack of imaging systems targeted specifically at plant science duties, resulting in low precision for canopy-scale measurements. This study trials a prototype multispectral system designed specifically for plant studies and looks at its use as an early detection system for visually asymptomatic disease phases, in this case Pyrenopeziza brassicae in Brassica napus. The analysis takes advantage of machine learning in the form of feature selection and novelty detection to facilitate the classification. An initial study into recording the morphology of the samples is also included to allow for further improvement to the system performance. Results: The proposed method was able to detect light leaf spot infection with 92% accuracy when imaging entire oilseed rape plants from above, 12 days after inoculation and 13 days before the appearance of visible symptoms. False colour mapping of spectral vegetation indices was used to quantify disease severity and its distribution within the plant canopy. In addition, the structure of the plant was recorded using photometric stereo, with the output influencing regions used for diagnosis. The shape of the plants was also recorded using photometric stereo, which allowed for reconstruction of the leaf angle and surface texture, although further work is needed to improve the fidelity due to uneven lighting distributions, to allow for reflectance compensation. Conclusions: The ability of active multispectral imaging has been demonstrated along with the improvement in time taken to detect light leaf spot at a high accuracy. The importance of capturing structural information is outlined, with its effect on reflectance and thus classification illustrated. The system could be used in plant breeding to enhance the selection of resistant cultivars, with its early and quantitative capability. [ABSTRACT FROM AUTHOR]