Search

This paper investigates the system identification problem for linear discrete-time systems under adversaries and analyzes two lasso-type estimators. We examine non-asymptotic properties of these estimators in two separate scenarios, corresponding to deterministic and stochastic models for the attack times. We prove that when the system is stable and attacks are injected periodically, the sample complexity for exact recovery of the system dynamics is linear in terms of the dimension of the states. When adversarial attacks occur at each time instance with probability $p$, the required sample complexity for exact recovery scales polynomially in the dimension of the states and the probability $p$. This result implies almost sure convergence to the true system dynamics under the asymptotic regime. As a by-product, our estimators still learn the system correctly even when more than half of the data is compromised. We emphasize that the attack vectors are allowed to be correlated with each other in this work. This paper provides the first mathematical guarantee in the literature on learning from correlated data for dynamical systems in the case when there is less clean data than corrupt data.

This research investigates the potential of in vivo learning to enhance visual representation learning for image-based person re-identification (re-ID). Compared to traditional self-supervised learning (which require external data), the introduced in vivo learning utilizes supervisory labels generated from pedestrian images to improve re-ID accuracy without relying on external data sources. Three carefully designed in vivo learning tasks, leveraging statistical regularities within images, are proposed without the need for laborious manual annotations. These tasks enable feature extractors to learn more comprehensive and discriminative person representations by jointly modeling various aspects of human biological structure information, contributing to enhanced re-ID performance. Notably, the method seamlessly integrates with existing re-ID frameworks, requiring minimal modifications and no additional data beyond the existing training set. Extensive experiments on diverse datasets, including Market1501, CUHK03-NP, Celeb-reID, Celeb-reid-light, PRCC, and LTCC, demonstrate substantial enhancements in rank-1 precision compared to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Online Social Media (OSM) are dominating the wide range of Internet services. Due to their vast audience, it is crucial to evaluate the interpersonal trust among OSM users that can identify reliable sources of information, the meaningfulness of a relationship, or the trustworthiness of other users. SentiTrust is an innovative trust model for Decentralized Online Social Networks that is based on AI-powered Sentiment Analysis. It enriches the trust definition by exploiting important features that are enabled because of the adoption of Social Media through mobile devices. The model can be easily extended and customized according to the scenario of interest. The sentiment analysis component has been tested by involving 30 participants who completed several guided tasks using a social media application while their electrodermal activity and rate responses were measured. The results suggest that low arousal states are related to receiving happy faces and to sending more messages per minute. Furthermore, positive interactions result in shorter interactions and multimedia exchanges.

In the application scenarios of radio frequency identification technology, there are many situations where a large number of labels respond to the reader at the same time, resulting in the labels not being able to be identified for a long time. In order to address the label collision problem of radio frequency identification, this paper studies the impact of statistical learning method on the resolution and decoding of collision labels, and proposes a novel clustering method using maximum posteriori probability estimation based on Monte-Carlo. Unlike traditional algorithms, the proposed method does not require prior knowledge of the number of clusters and does not need to constantly iterate. In addition, this method has low complexity and ensures both accuracy and robustness while quickly finding the cluster centroids. Finally, the resolution performance of the proposed method is evaluated based on the simulation experiment and the field experiment, and the resolved signals are decoded using matched filter and phase jump. Overall, the effectiveness of the our method is demonstrated through comparisons with different performance metrics of different benchmark methods, including bit error rate, resolution efficiency, throughput, error, and time complexity.

Cropland abandonment is crucial in agricultural management and has a profound impact on crop yield and food security. In recent years, many cropland abandonment identification methods based on remote sensing observation data have been proposed, but most of these methods are based on coarse-resolution images and use traditional machine learning methods for simple identification. To this end, we perform abandonment recognition on high-resolution remote sensing images. According to the texture features of the abandoned land, we combine the method of statistical texture learning and propose a new deep learning framework called pyramid scene parsing network-statistical texture learning (PSPNet-STL). The model integrates high-level semantic feature extraction and deep mining of low-level texture features to identify cropland abandonment. First, we labeled the abandoned cropland area and built the high-resolution abandoned cropland (HRAC) dataset, a high-resolution cropland abandonment dataset. Second, we improved PSPNet by fusing statistical texture learning modules to learn multiple texture information on low-level feature maps and combined high-level semantic features for cropland abandonment recognition. Experiments are performed on the HRAC dataset. Compared with other methods, the proposed model has the best performance on this dataset, both in terms of accuracy and visualization, proving that deep mining of low-level statistical texture features is beneficial for crop abandonment recognition.

Image deblurring and its counterpart blind problem are undoubtedly two fundamental tasks in computational imaging and computer vision. Interestingly, deterministic edge-preserving regularization for maximum-a-posteriori (MAP) based non-blind image deblurring has been largely made clear 25 years ago. As for the blind task, the state-of-the-art MAP-based approaches seem to also reach a consensus on the characteristic of deterministic image regularization, i.e., formulated in an L0 composite style or termed as L0+X style, where X is often a discriminative term such as dark channels-based sparsity regularization. However, with a modeling perspective as such, non-blind and blind deblurring are entirely disconnected from each other. Additionally, because L0 and X are motivated very differently in general, it is not easy in practice to derive an efficient numerical scheme. In fact, since the prosperity of modern blind deblurring 15 years ago, a physically intuitive yet practically effective and efficient regularization has been always desired. In this paper, representative deterministic image regularization terms in MAP-based blind deblurring are firstly revisited, with an emphasis on their differences from edge-preserving regularization for non-blind deblurring. Inspired by existing robust losses in the statistical and deep learning literature, an insightful conjecture is then made. That is, deterministic image regularization for blind deblurring can be naively formulated using a type of redescending potential functions (RDP), and interestingly, a RDP-induced blind deblurring regularization term is actually the $1^{rst}$ -order derivative of a nonconvex edge-preserving regularization for non-blind image deblurring. An intimate relationship in regularization is therefore established between the two problems, differing much from the mainstream modeling perspective on blind deblurring. Via above principle analysis, the conjecture is demonstrated on benchmark deblurring problems in the final, accompanied with comparisons against several top-performing L0+X style methods. We note that, the rationality and practicality of the RDP-induced regularization is particularly highlighted here, aiming to open up an alternative line of possibility for modeling blind deblurring. [ABSTRACT FROM AUTHOR]

This article aims to determine factors crucial for the performance of new ventures by performing a meta-analysis. It evaluates the performance of machine learning and statistical models by selecting 19 studies through preferred reporting items for systematic reviews and meta-analyses (PRISMA). The results obtained were graphically represented using forest plots. A subjective analysis of the studies revealed “business plan,” “market scope,” “team size,” “service timing,” “market growth,” and “age of the entrepreneur” as crucial factors for deciding the performance of the new ventures. In machine learning model-based studies, the combined model was found to be statistically significant. The variables, namely, “seed funding,” “funding rounds,” “location,” “social media presence,” “team size,” “number of founding members,” and “defunct date” affect the predictive capability of these models. The implications of this article can be applied by start-ups to reduce uncertainty and enhance performance. [ABSTRACT FROM AUTHOR]

Seated postural limit defines the boundary of a region such that for any excursions made outside this boundary a subject cannot return the trunk to the neutral position without additional external support. The seated postural limits can be used as a reference to provide assistive support to the torso by the Trunk Support Trainer (TruST). However, fixed boundary representations of seated postural limits are inadequate to capture dynamically changing seated postural limits during training. In this study, we propose a conceptual model of dynamic boundary of the trunk center by assigning a vector that tracks the postural-goal direction and trunk movement amplitude during a sitting task. We experimented with 20 healthy subjects. The results support our hypothesis that TruST intervention with an assist-as-needed force controller based on dynamic boundary representation could achieve more significant sitting postural control improvements than a fixed boundary representation. The second contribution of this paper is that we provide an effective approach to embed deep learning into TruST’s real-time controller design. We have compiled a 3D trunk movement dataset which is currently the largest in the literature. We designed a loss function capable of solving the gate-controlled regression problem. We have proposed a novel deep-learning roadmap for the exploration study. Following the roadmap, we developed a deep learning architecture, modified the widely used Inception module, and then obtained a deep learning model capable of accurately predicting the dynamic boundary in real-time. We believe that this approach can be extended to other rehabilitation robots towards designing intelligent dynamic boundary-based assist-as-needed controllers. [ABSTRACT FROM AUTHOR]

The KC-135R Stratotanker is a multifunction slim body aircraft that provides air refueling and airlift for the United States’ war and peacetime requirements, which demand a certain level of availability. As the KC-135R fleet ages, the aircraft availability (AA) rate degrades due to high demand use, stress, and the age of the equipment. Preventative and corrective maintenance is designed to return the aircraft to an available state to meet mission requirements, but the United States Air Force continues to fail at meeting every requirement communicated by commanders for KC-135R air refueling and airlift. Focusing on aircraft metrics can enable a prediction model of AA and provide the unit commanders the tools for data influenced decisions. The analysis of historical aircraft maintenance and flight metrics will show a correlation between tracked metrics and AA, thus, the ability to predict a future availability rate. Furthermore, analyzing the data with machine learning techniques will improve prediction accuracy by evaluating the variable importance and will make inferences learned from mining the data that are otherwise difficult to model in a complex system of systems. [ABSTRACT FROM AUTHOR]

The generally accepted view among managers and researchers is of a direct linear relationship between service quality and profitability. Supported by the strategy-structure-paradigm, we argue that the relationship between service quality and organizational performance is impacted by strategy. Using historical data from the U.S. airline industry, we study the impact of various service failures on financial performance between airlines pursuing different strategies (low cost versus legacy). Using a novel methodological approach that includes neural network modeling and statistical data analysis, we find that the impact of service failures on profitability is not linear but differs between the two groups across all measures (flight delays, involuntary denied boarding, and mishandled baggage). Most interestingly, it is the pattern of the relationship that is distinctly different. These findings reveal the nuanced impact that service failure has on profitability. Although customer complaints are found to have the most significant impact, the pattern of impact is not linear and is distinctly different between the two groups. These findings challenge the conventional belief that reducing customer complaints goes hand in hand with improving financial performance. The results provide significant managerial implications and contribute to the existing body of knowledge on service quality and operations strategy. [ABSTRACT FROM AUTHOR]

Exabytes of data are generated daily by humans, leading to the growing needs for new efforts in dealing with the grand challenges for multi-label learning brought by big data. For example, extreme multi-label classification is an active and rapidly growing research area that deals with classification tasks with extremely large number of classes or labels; utilizing massive data with limited supervision to build a multi-label classification model becomes valuable for practical applications, etc. Besides these, there are tremendous efforts on how to harvest the strong learning capability of deep learning to better capture the label dependencies in multi-label learning, which is the key for deep learning to address real-world classification tasks. However, it is noted that there have been a lack of systemic studies that focus explicitly on analyzing the emerging trends and new challenges of multi-label learning in the era of big data. It is imperative to call for a comprehensive survey to fulfil this mission and delineate future research directions and new applications. [ABSTRACT FROM AUTHOR]

Neural networks have achieved remarkable successes in machine learning tasks. This has recently been extended to graph learning using neural networks. However, there is limited theoretical work in understanding how and when they perform well, especially relative to established statistical learning techniques such as spectral embedding. In this short paper, we present a simple generative model where unsupervised graph convolutional network fails, while the adjacency spectral embedding succeeds. Specifically, unsupervised graph convolutional network is unable to look beyond the first eigenvector in certain approximately regular graphs, thus missing inference signals in non-leading eigenvectors. The phenomenon is demonstrated by visual illustrations and comprehensive simulations. [ABSTRACT FROM AUTHOR]

Radar-based gesture recognition systems are a promising concept for new human–machine interfaces. However, the systems are sensitive to user-dependent gesture patterns, sensor noise characteristics, background environments, and unknown gestures or background motions. To overcome such challenges, we propose a variational autoencoder architecture-based deep metric learning, which is optimized using a novel loss function combining a statistical distance triplet loss and the center loss. By learning the statistical distance between distributions, the proposed solution, compared with triplet loss, is able to better learn the nonlinear characteristics in the data, is less sensitivity to training strategy, and is capable of creating close knit embedding clusters. Moreover, it is experimentally demonstrated that the proposed gesture recognition system using radar spectrograms has improved classification accuracy and random gesture rejection accuracy compared with the state-of-the-art deep metric learning approaches. Furthermore, we proof the real-world capability of the system by implementing a real-time demonstrator, which allows playing “Tetris” with hand gestures. [ABSTRACT FROM AUTHOR]

A connection between the general linear model (GLM) with frequentist statistical testing and machine learning (MLE) inference is derived and illustrated. Initially, the estimation of GLM parameters is expressed as a Linear Regression Model (LRM) of an indicator matrix; that is, in terms of the inverse problem of regressing the observations. Both approaches, i.e. GLM and LRM, apply to different domains, the observation and the label domains, and are linked by a normalization value in the least-squares solution. Subsequently, we derive a more refined predictive statistical test: the linear Support Vector Machine (SVM), that maximizes the class margin of separation within a permutation analysis. This MLE-based inference employs a residual score and associated upper bound to compute a better estimation of the actual (real) error. Experimental results demonstrate how parameter estimations derived from each model result in different classification performance in the equivalent inverse problem. Moreover, using real data, the MLE-based inference including model-free estimators demonstrates an efficient trade-off between type I errors and statistical power. [ABSTRACT FROM AUTHOR]

Dilute combustion using exhaust gas recirculation (EGR) presents a cost-effective method for increasing the efficiency of spark-ignition (SI) engines. However, the maximum amount of EGR that can be used at a given condition is limited by a rapid increment of cycle-to-cycle variability (CCV). This study describes a methodology to design a model-based stochastic optimal controller to adjust the cycle-to-cycle fuel injection quantity in order to reduce CCV and further extend the dilute limit. Given the complexity and chaotic nature of combustion events, the controller was enhanced with online learning in order to identify the statistical properties of combustion efficiency, which are needed to generate predictions for next-cycle events. This study showed that a kernel density estimator (KDE) can be used to learn the combustion properties in real time and can be incorporated into the feedback policy in order to calculate the optimal control command. Experimental results suggested that the dilute limit can be extended from 18.5% to 21% EGR fraction at an operating condition relevant for highway cruising. Additionally, the proposed controller can achieve a large CCV reduction with less fuel enrichment compared to previous methods, overall contributing to an increase in 0.2% indicated fuel conversion efficiency. [ABSTRACT FROM AUTHOR]

Oscillation-based testing of analogue electronic filters removes the need for test signal synthesis. Parametric faults in the presence of normal component tolerance variation are challenging to detect and diagnose. This study demonstrates the suitability of statistical learning and deep learning techniques for parametric fault diagnosis and detection by investigating several time-series classification techniques. Traditional harmonic analysis is used as a baseline for an in-depth comparison. Eight standard classification techniques are applied and compared. Deep learning approaches, which classify the time-series signals directly, are shown to benefit from the oscillator start-up region for feature extraction. Global average pooling in the convolutional neural networks (CNN) allows for Class Activation Maps (CAM). This enables interpreting the time-series signal’s discriminative regions and confirming the importance of the start-up oscillation signal. The deep learning approach outperforms the harmonic analysis approach on simulated data by an average of 11.77% in classification accuracy for the three parametric fault magnitudes considered in this work.

In this paper, we present a Model-Free Stochastic Inverse Optimal Control (IOC) algorithm for the discrete-time infinite-horizon stochastic linear quadratic regulator (LQR). Our proposed algorithm exploits the richness of the available system trajectories to recover the control gain $K$ and cost function parameters $(Q,R)$ in a low (space, sample, and computational) complexity manner. By leveraging insights on the stochastic LQR, we guarantee well-posedness of the Model-Free Stochastic IOC LQR via satisfaction of the Certainty Equivalence optimality conditions. The exact solution of the control gain $K$ is recovered via a deterministic, low complexity Least Squares approach. Using $K$ , we solve a completely model-free non-iterative SemiDefinite Programming (SDP) problem to obtain a unique (up to a scalar ambiguity) $(Q,R)$ , in which optimality and feasibility are jointly ensured. Via derivation of the sample complexity bounds, we show that the non-asymptotic performance of the Model-Free Stochastic IOC LQR can be characterized by the signal-to-noise (SNR) ratio of the finite set of system state and input signals. We present a model-based version of the algorithm for the special case where $(A,B)$ is available, and we, further, provide the extension to the Stochastic Model-Free IOC linear quadratic tracking (LQT) case.

Networked control systems have gained considerable attention over the last decade as a result of the trend towards decentralised control applications and the emergence of cyber-physical system applications. However, real-world wireless networked control systems suffer from limited communication bandwidths, reliability issues, and a lack of awareness of network dynamics due to the complex nature of wireless networks. Combining machine learning and event-triggered control has the potential to alleviate some of these issues. For example, machine learning can be used to overcome the problem of a lack of network models by learning system behavior or adapting to dynamically changing models by continuously learning model dynamics. Event-triggered control can help to conserve communication bandwidth by transmitting control information only when necessary or when resources are available. The purpose of this article is to conduct a review of the literature on the use of machine learning in combination with event-triggered control. Machine learning techniques such as statistical learning, neural networks, and reinforcement learning-based approaches such as deep reinforcement learning are being investigated in combination with event-triggered control. We discuss how these learning algorithms can be used for different applications depending on the purpose of the machine learning use. Following the review and discussion of the literature, we highlight open research questions and challenges associated with machine learning-based event-triggered control and suggest potential solutions.

In this work, we analyze the finite sample complexity bounds for offline reinforcement learning with general state, general function space and state-dependent action sets. The algorithm analyzed does not require the knowledge of the data-collection policy as compared to earlier works. We show that one can compute an $\epsilon$-optimal Q function (state-action value function) using $O(1/\epsilon ^{4})$ i.i.d. samples of state-action-reward-next state tuples.

The foundational assumption of machine learning is that the data under consideration is separable into classes; while intuitively reasonable, separability constraints have proven remarkably difficult to formulate mathematically. We believe this problem is rooted in the mismatch between existing statistical techniques and commonly encountered data; object representations are typically high dimensional but statistical techniques tend to treat high dimensions a degenerate case. To address this problem, we develop a dedicated statistical framework for machine learning in high dimensions. The framework derives from the observation that object relations form a natural hierarchy; this leads us to model objects as instances of a high dimensional, hierarchal generative processes. Using a distance based statistical technique, also developed in this paper, we show that in such generative processes, instances of each process in the hierarchy, are almost-always encapsulated by a distinctive-shell that excludes almost-all other instances. The result is shell theory, a statistical machine learning framework in which separability constraints (distinctive-shells) are formally derived from the assumed generative process. [ABSTRACT FROM AUTHOR]

In this paper, we analyze the generalization performance of the Iterative Hard Thresholding (IHT) algorithm widely used for sparse recovery problems. The parameter estimation and sparsity recovery consistency of IHT has long been known in compressed sensing. From the perspective of statistical learning, another fundamental question is how well the IHT estimation would predict on unseen data. This paper makes progress towards answering this open question by introducing a novel sparse generalization theory for IHT under the notion of algorithmic stability. Our theory reveals that: 1) under natural conditions on the empirical risk function over $n$ samples of dimension $p$ , IHT with sparsity level $k$ enjoys an $\tilde {\mathcal {O}}(n^{-1/2}\sqrt {k\log (n)\log (p)})$ rate of convergence in sparse excess risk; 2) a tighter $\tilde {\mathcal {O}}(n^{-1/2}\sqrt {\log (n)})$ bound can be established by imposing an additional iteration stability condition on a hypothetical IHT procedure invoked to the population risk; and 3) a fast rate of order $\tilde {\mathcal {O}}\left ({n^{-1}k(\log ^{3}(n)+\log (p))}\right)$ can be derived for strongly convex risk function under proper strong-signal conditions. The results have been substantialized to sparse linear regression and sparse logistic regression models to demonstrate the applicability of our theory. Preliminary numerical evidence is provided to support our theoretical predictions. [ABSTRACT FROM AUTHOR]

Recent domain adaptation works rely on an indirect way of first aligning the source and target domain distributions and then train a classifier on the labeled source domain to classify the target domain. However, the main drawback of this approach is that obtaining a near-perfect domain alignment in itself might be difficult/impossible (e.g., language domains). To address this, inspired by how humans use supervised-unsupervised learning to perform tasks seamlessly across multiple domains or tasks, we follow Vapnik’s imperative of statistical learning that states any desired problem should be solved in the most direct way rather than solving a more general intermediate task and propose a direct approach to domain adaptation that does not require domain alignment. We propose a model referred to as Contradistinguisher that learns contrastive features and whose objective is to jointly learn to contradistinguish the unlabeled target domain in an unsupervised way and classify in a supervised way on the source domain. We achieve the state-of-the-art on Office-31, Digits and VisDA-2017 datasets in both single-source and multi-source settings. We demonstrate that performing data augmentation results in an improvement in the performance over vanilla approach. We also notice that the contradistinguish-loss enhances performance by increasing the shape bias. [ABSTRACT FROM AUTHOR]

Objective: Statistical shape models (SSMs) are a popular tool to conduct morphological analysis of anatomical structures which is a crucial step in clinical practices. However, shape representations through SSMs are based on shape coefficients and lack an explicit one-to-one relationship with anatomical measures of clinical relevance. While a shape coefficient embeds a combination of anatomical measures, a formalized approach to find the relationship between them remains elusive in the literature. This limits the use of SSMs to subjective evaluations in clinical practices. We propose a novel SSM controlled by anatomical parameters derived from morphometric analysis. Methods: The proposed anatomically parameterized SSM (ANAT $_{\text{SSM}}$) is based on learning a linear mapping between shape coefficients (latent space) and selected anatomical parameters (anatomical space). This mapping is learned from a synthetic population generated by the standard SSM. Determining the pseudo-inverse of the mapping allows us to build the ANAT $_{\text{SSM}}$. We further impose orthogonality constraints to the anatomical parameterization (OC-ANAT $_{\text{SSM}}$) to obtain independent shape variation patterns. The proposed contribution was evaluated on two skeletal databases of femoral and scapular bone shapes using clinically relevant anatomical parameters within each (five for femoral and six for scapular bone). Results: Anatomical measures of the synthetically generated shapes exhibited realistic statistics. The learned matrices corroborated well with the obtained statistical relationship, while the two SSMs achieved moderate to excellent performance in predicting anatomical parameters on unseen shapes. Conclusion: This study demonstrates the use of anatomical representation for creating anatomically parameterized SSMs and as a result, removes the limited clinical interpretability of standard SSMs. Significance: The proposed models could help analyze differences in relevant bone morphometry between populations, and be integrated in patient-specific pre-surgery planning or in-surgery assessment. [ABSTRACT FROM AUTHOR]

Following the roadmap of carbon neutrality, wireless communication systems are upgrading to use green energy that comes from renewable sources, e.g., sun, tide, and wind. Due to the volatile arrival of green energy, the on-grid energy is used as a backup for a green coordinated multiple point system. In this work, a weighted sum-rate maximization problem in the green coordinated multiple point system is investigated by expecting non-positive consumption of the on-grid energy in the long term. Motivated by the capacity-achieving property and simple implementation, an online zero-forcing dirty paper precoder is proposed to update the precoding matrices by combining statistical learning with the Lyapunov learning technique. A tradeoff relation is theoretically established to show that the long-term weighted sum rate approaches the ${\mathcal{ O}}(V)$ -neighbor of optimal value while the long-term on-grid energy increases at a rate of ${\mathcal{ O}}({\scriptstyle {}^{\scriptstyle \log ^{2}(V)}}\hspace {-0.224em}/\hspace {-0.112em}{\scriptstyle \sqrt {V}})$ , where $V$ is an introduced control parameter. Numerical results are used to verify the performance of the proposed online adaptive precoder. [ABSTRACT FROM AUTHOR]

The fractional hot-deck imputation (FHDI) is a general-purpose, assumption-free imputation method for handling multivariate missing data by filling each missing item with multiple observed values without resorting to artificially created values. The corresponding R package FHDI J. Im, I. Cho, and J. K. Kim, “An R package for fractional hot deck imputation,” R J., vol. 10, no. 1, pp. 140–154, 2018 holds generality and efficiency, but it is not adequate for tackling big incomplete data due to the requirement of excessive memory and long running time. As a first step to tackle big incomplete data by leveraging the FHDI, we developed a new version of a parallel fractional hot-deck imputation (named as P-FHDI) program suitable for curing large incomplete datasets. Results show a favorable speedup when the P-FHDI is applied to big datasets with up to millions of instances or 10,000 of variables. This paper explains the detailed parallel algorithms of the P-FHDI for large instances (big- $n$ n ) or high-dimensionality (big- $p$ p ) datasets and confirms the favorable scalability. The proposed program inherits all the advantages of the serial FHDI and enables a parallel variance estimation, which will benefit a broad audience in science and engineering. [ABSTRACT FROM AUTHOR]

The increase in distributed energy resources and flexible electricity consumers has turned TSO-DSO coordination strategies into a challenging problem. Existing decomposition/decentralized methods apply divide-and-conquer strategies to trim down the computational burden of this complex problem, but rely on access to proprietary information or fail-safe real-time communication infrastructures. To overcome these drawbacks, we propose in this paper a TSO-DSO coordination strategy that only needs a series of observations of the nodal price and the power intake at the substations connecting the transmission and distribution networks. Using this information, we learn the price response of active distribution networks (DN) using a decreasing step-wise function that can also adapt to some contextual information. The learning task can be carried out in a computationally efficient manner and the curve it produces can be interpreted as a market bid, thus averting the need to revise the current operational procedures for the transmission network. Inaccuracies derived from the learning task may lead to suboptimal decisions. However, results from a realistic case study show that the proposed methodology yields operating decisions very close to those obtained by a fully centralized coordination of transmission and distribution. [ABSTRACT FROM AUTHOR]

In this paper we show how to perform scene-level inverse rendering to recover shape, reflectance and lighting from a single, uncontrolled image using a fully convolutional neural network. The network takes an RGB image as input, regresses albedo, shadow and normal maps from which we infer least squares optimal spherical harmonic lighting coefficients. Our network is trained using large uncontrolled multiview and timelapse image collections without ground truth. By incorporating a differentiable renderer, our network can learn from self-supervision. Since the problem is ill-posed we introduce additional supervision. Our key insight is to perform offline multiview stereo (MVS) on images containing rich illumination variation. From the MVS pose and depth maps, we can cross project between overlapping views such that Siamese training can be used to ensure consistent estimation of photometric invariants. MVS depth also provides direct coarse supervision for normal map estimation. We believe this is the first attempt to use MVS supervision for learning inverse rendering. In addition, we learn a statistical natural illumination prior. We evaluate performance on inverse rendering, normal map estimation and intrinsic image decomposition benchmarks. [ABSTRACT FROM AUTHOR]

This paper proposes a novel pretext task to address the self-supervised video representation learning problem. Specifically, given an unlabeled video clip, we compute a series of spatio-temporal statistical summaries, such as the spatial location and dominant direction of the largest motion, the spatial location and dominant color of the largest color diversity along the temporal axis, etc. Then a neural network is built and trained to yield the statistical summaries given the video frames as inputs. In order to alleviate the learning difficulty, we employ several spatial partitioning patterns to encode rough spatial locations instead of exact spatial Cartesian coordinates. Our approach is inspired by the observation that human visual system is sensitive to rapidly changing contents in the visual field, and only needs impressions about rough spatial locations to understand the visual contents. To validate the effectiveness of the proposed approach, we conduct extensive experiments with four 3D backbone networks, i.e., C3D, 3D-ResNet, R(2+1)D and S3D-G. The results show that our approach outperforms the existing approaches across these backbone networks on four downstream video analysis tasks including action recognition, video retrieval, dynamic scene recognition, and action similarity labeling. The source code is publicly available at: https://github.com/laura-wang/video_repres_sts. [ABSTRACT FROM AUTHOR]

Variational representations of divergences and distances between high-dimensional probability distributions offer significant theoretical insights and practical advantages in numerous research areas. Recently, they have gained popularity in machine learning as a tractable and scalable approach for training probabilistic models and for statistically differentiating between data distributions. Their advantages include: 1) They can be estimated from data as statistical averages. 2) Such representations can leverage the ability of neural networks to efficiently approximate optimal solutions in function spaces. However, a systematic and practical approach to improving the tightness of such variational formulas, and accordingly accelerate statistical learning and estimation from data, is currently lacking. Here we develop such a methodology for building new, tighter variational representations of divergences. Our approach relies on improved objective functionals constructed via an auxiliary optimization problem. Furthermore, the calculation of the functional Hessian of objective functionals unveils the local curvature differences around the common optimal variational solution; this quantifies and orders the tightness gains between different variational representations. Finally, numerical simulations utilizing neural network optimization demonstrate that tighter representations can result in significantly faster learning and more accurate estimation of divergences in both synthetic and real datasets (of more than 1000 dimensions), often accelerated by nearly an order of magnitude. [ABSTRACT FROM AUTHOR]

This paper presents a novel data-driven framework to aid in system state estimation when the power system is under unobservable false data injection attacks. The proposed framework dynamically detects and classifies false data injection attacks. Then, it retrieves the control signal using the acquired information. This process is accomplished in three main modules, with novel designs, for detection, classification, and control signal retrieval. The detection module monitors historical changes in phasor measurements and captures any deviation pattern caused by an attack on a complex plane. This approach can help to reveal characteristics of the attacks including the direction, magnitude, and ratio of the injected false data. Using this information, the signal retrieval module can easily recover the original control signal and remove the injected false data. Further information regarding the attack type can be obtained through the classifier module. The proposed ensemble learner is compatible with harsh learning conditions including the lack of labeled data, concept drift, concept evolution, recurring classes, and independence from external updates. The proposed novel classifier can dynamically learn from data and classify attacks under all these harsh learning conditions. The introduced framework is evaluated w.r.t. real-world data captured from the Central New York Power System. The obtained results indicate the efficacy and stability of the proposed framework. [ABSTRACT FROM AUTHOR]

This work considers low-rank canonical polyadic decomposition (CPD) under a class of non-Euclidean loss functions that frequently arise in statistical machine learning and signal processing. These loss functions are often used for certain types of tensor data, e.g., count and binary tensors, where the least squares loss is considered unnatural. Compared to the least squares loss, the non-Euclidean losses are generally more challenging to handle. Non-Euclidean CPD has attracted considerable interests and a number of prior works exist. However, pressing computational and theoretical challenges, such as scalability and convergence issues, still remain. This work offers a unified stochastic algorithmic framework for large-scale CPD decomposition under a variety of non-Euclidean loss functions. Our key contribution lies in a tensor fiber sampling strategy-based flexible stochastic mirror descent framework. Leveraging the sampling scheme and the multilinear algebraic structure of low-rank tensors, the proposed lightweight algorithm ensures global convergence to a stationary point under reasonable conditions. Numerical results show that our framework attains promising non-Euclidean CPD performance. The proposed framework also exhibits substantial computational savings compared to state-of-the-art methods. [ABSTRACT FROM AUTHOR]

We consider the broad class of decentralized optimal resource allocation problems in wireless networks, which can be formulated as a constrained statistical learning problems with a localized information structure. We develop the use of Aggregation Graph Neural Networks (Agg-GNNs), which process a sequence of delayed and potentially asynchronous graph aggregated state information obtained locally at each transmitter from multi-hop neighbors. We further utilize model-free primal-dual learning methods to optimize performance subject to constraints in the presence of delay and asynchrony inherent to decentralized networks. We demonstrate a permutation equivariance property of the resulting resource allocation policy that can be shown to facilitate transference to dynamic network configurations. The proposed framework is validated with numerical simulations that exhibit superior performance to baseline strategies. [ABSTRACT FROM AUTHOR]

We study online active learning for classifying streaming instances within the framework of statistical learning theory. At each time, the learner either queries the label of the current instance or predicts the label based on past seen examples. The objective is to minimize the number of queries while constraining the number of prediction errors over a horizon of length $T$. We develop a disagreement-based online learning algorithm for a general hypothesis space and under the Tsybakov noise and establish its label complexity under a constraint of bounded regret in terms of classification errors. We further establish a matching (up to a poly-logarithmic factor) lower bound, demonstrating the order optimality of the proposed algorithm. We address the tradeoff between label complexity and regret and show that the algorithm can be modified to operate at a different point on the tradeoff curve. [ABSTRACT FROM AUTHOR]

Channel multipath components (MPCs) clustering and cluster characterization are the prerequisite for the development of cluster based channel models. This article investigates the MPCs clustering based on machine learning (ML) and analyzes the cluster characteristics in typical high-speed railway (HSR) scenarios. A variational Bayesian Gaussian mixture model (GMM)-based algorithm is introduced to achieve the space–time clustering of MPCs, which incorporates the statistical characteristics of MPCs and can automatically determine the optimum number of clusters. Moreover, a novel density-based validity index is proposed for evaluating the MPCs clustering performance. The proposed validity index improves the traditional index by considering the intracluster density, which can be calculated according to the Graham scanning method and Green’s formula. In addition to synthetic datasets, realistic MPCs datasets collected in an HSR obstructed viaduct scenario are used for the performance evaluation of the clustering algorithm and clustering validity index. Based on clustering results in the measured scenario, static cluster characteristics, including cluster number, intercluster and intracluster delay spreads (DSs) and angle spreads (ASs), and dynamic cluster characteristics such as cluster lifetime and birth and death property of clusters, are extracted and analyzed. These results will be useful for cluster-based channel modeling in future HSR mobile communication systems. [ABSTRACT FROM AUTHOR]

Due to the unique advantages of long-distance, non-relay, and flexible deployment, high frequency (HF) communication plays a vital role in military communication, disaster relief, and global broadcasting, etc. To satisfying the spectrum planning requirement for the next generation intelligent HF communication system, the machine learning method is introduced to develop the long-term prediction method of the usable working frequency for HF wireless communication. Especially, the refined mapping model of maximum usable frequency (MUF) propagation factor for one hop on the F2 layer is first reconstructed by using the statistical machine learning method. Then, the new mapping models of conversion factors of optimum working frequency (OWF) and the highest probable frequency (HPF) are proposed by using the fine-grained solar activity parameters and coupling with two geomagnetic activity parameters. This proposed model has higher prediction accuracy for the MUF, OWF, and HPF over Asia. Compared with the International Telecommunication Union (ITU) recommended model, the root-mean-square errors of MUF, OWF, and HPF are reduced by 1.18, 1.64, and 1.06 MHz, and the accuracies are improved by 10.89%, 15.47%, and 9.10%, respectively. The proposed model can achieve intelligent frequency planning for HF communication and has great potential in terms of improving HF communication quality, reliability, and efficiency. [ABSTRACT FROM AUTHOR]

Learning generative probabilistic models is a core problem in machine learning, which presents significant challenges due to the curse of dimensionality. This paper proposes a joint dimensionality reduction and non-parametric density estimation framework, using a novel estimator that can explicitly capture the underlying distribution of appropriate reduced-dimension representations of the input data. The idea is to jointly design a nonlinear dimensionality reducing auto-encoder to model the training data in terms of a parsimonious set of latent random variables, and learn a canonical low-rank tensor model of the joint distribution of the latent variables in the Fourier domain. The proposed latent density model is non-parametric and “universal,” as opposed to the predefined prior that is assumed in variational auto-encoders. Joint optimization of the auto-encoder and the latent density estimator is pursued via a formulation which learns both by minimizing a combination of the negative log-likelihood in the latent domain and the auto-encoder reconstruction loss. We demonstrate that the proposed model achieves very promising results on toy, tabular, and image datasets on regression tasks, sampling, and anomaly detection. [ABSTRACT FROM AUTHOR]

Effective non-parametric density estimation is a key challenge in high-dimensional multivariate data analysis. In this paper, we propose a novel approach that builds upon tensor factorization tools. Any multivariate density can be represented by its characteristic function, via the Fourier transform. If the sought density is compactly supported, then its characteristic function can be approximated, within controllable error, by a finite tensor of leading Fourier coefficients, whose size depends on the smoothness of the underlying density. This tensor can be naturally estimated from observed and possibly incomplete realizations of the random vector of interest, via sample averaging. In order to circumvent the curse of dimensionality, we introduce a low-rank model of this characteristic tensor, which significantly improves the density estimate especially for high-dimensional data and/or in the sample-starved regime. By virtue of uniqueness of low-rank tensor decomposition, under certain conditions, our method enables learning the true data-generating distribution. We demonstrate the very promising performance of the proposed method using several toy, measured, and image datasets. [ABSTRACT FROM AUTHOR]

Mission-critical Internet-of-Things (IoT) applications require communication interfaces that provide ultra-reliability and low latency. Acquiring knowledge regarding the number of active devices and their latency-reliability requirements becomes essential to optimize resource allocation in heterogeneous networks. Due to the inherent heavy computation overheads, the conventional centralized decision-making approaches result in large latency. The distributed computing and device-level prediction of network parameters can play a significant role in designing mission-critical IoT applications operating in dynamic environments. This paper considers the medium access control (MAC) layer of heterogeneous networks employing a framed-ALOHA-based restricted transmission strategy to enhance reliability. We present a statistical learning-based device-level network exploration mechanism in which end-devices use their transmission history to predict different network parameters. The IoT devices share the learned parameters with the base station (BS) to identify different groups presented in the network. The simulation results show that the mean square error (MSE) in predicting different network parameters can be reduced by increasing the history window size. In this regard, the optimal size of the history window under the given accuracy constraints is also determined. We demonstrate that the proposed device-level network load prediction mechanism is more robust as compared to the BS-centered approach. [ABSTRACT FROM AUTHOR]

We experience the power of machine learning (ML) in our everyday lives—be it picture and speech recognition, customized suggestions by virtual assistants, or just unlocking our phones. Its underlying mathematical principles have been applied since the middle of the last century in what is known as statistical learning. However, the enormous increase in computational power, even in devices as small as a smartphone, has enabled significant advances and wide adoption of ML in nearly every part of our lives and the scientific world. [ABSTRACT FROM AUTHOR]

Speech impairment in Parkinson’s Disease (PD) has been extensively studied. Our understanding of speech in people who are at an increased risk of developing PD is, however, rather limited. It is known that isolated Rapid Eye Movement (REM) sleep Behavior Disorder (RBD) is associated with a high risk of developing PD. The aim of this study is to investigate smartphone speech testing to: (1) distinguish participants with RBD from controls and PD, and (2) predict a range of self- or researcher-administered clinical scores that quantify participants’ motor symptoms, cognition, daytime sleepiness, depression, and the overall state of health. The rationale of our analyses is to test an initial hypothesis that speech can be used to detect and quantify the symptoms associated with RBD and PD. We analyzed 4242 smartphone voice recordings collected in clinic and at home from 92 Controls, 112 RBD and 335 PD participants. We used acoustic signal analysis and machine learning, employing 337 features that quantify different properties of speech impairment. Using a leave-one-subject-out cross-validation scheme, we were able to distinguish RBD from controls (sensitivity 60.7%, specificity 69.6%) and RBD from PD participants (sensitivity 74.9%, specificity 73.2%), and predict clinical assessments with clinically useful accuracy. These promising findings warrant further investigation in using speech as a digital biomarker for PD and RBD to facilitate intervention in the early and prodromal stages of PD.

This study investigates the relationship between two related computational thinking practices: data practices and computational problem-solving practices in acquiring related computational thinking practices during a first-year undergraduate engineering course. While computational thinking theory is still developing, empirical studies can help further understand how students demonstrate this knowledge and their progression in attaining the practices. Therefore, with this empirical study, the following questions are addressed. RQ1: What are the differences in students’ computational thinking practices at the beginning of an undergraduate introductory programming course? RQ 2: How do these differences correspond to the acquisition of more advanced computational thinking practices? The use of a descriptive non-experimental design that aims to understand the correlation between practices related to data and computational problem-solving is presented. A machine learning technique is employed, utilizing historical data from introductory programming for a problem-solving course with more than 1000 first-year engineering students. After identifying groups of students defining different profiles, data from posterior performance in more advanced programming topics were descriptively analyzed. This study supports the characterization of four different student profiles demonstrating differences in their performance at the beginning of the semester. From these four profiles, two of them show a subsequent differential progression besides their similarity at the beginning of the semester. In this particular case, troubleshooting and debugging appear as a relevant competency when distinguishing these two learners’ groups. These findings suggest that previous knowledge or exposure to different practices can result in different progressions of more complex computational practices, emphasizing the relevance of troubleshooting and debugging as a practice required for a successful and timely progression on the acquisition of other computational thinking practices.

This paper investigates the asymptotic behavior of the soft-margin and hard-margin support vector machine (SVM) classifiers for simultaneously high-dimensional and numerous data (large $n$ and large $p$ with $n/p\to \delta$) drawn from a Gaussian mixture distribution. Sharp predictions of the classification error rate of the hard-margin and soft-margin SVM are provided, as well as asymptotic limits of as such important parameters as the margin and the bias. As a further outcome, the analysis allows for the identification of the maximum number of training samples that the hard-margin SVM is able to separate. The precise nature of our results allows for an accurate performance comparison of the hard-margin and soft-margin SVM as well as a better understanding of the involved parameters (such as the number of measurements and the margin parameter) on the classification performance. Our analysis, confirmed by a set of numerical experiments, builds upon the convex Gaussian min-max Theorem, and extends its scope to new problems never studied before by this framework.

The autoencoder concept has fostered the reinterpretation and the design of modern communication systems. It consists of an encoder, a channel and a decoder block that modify their internal neural structure in an end-to-end learning fashion. However, the current approach to train an autoencoder relies on the use of the cross-entropy loss function. This approach can be prone to overfitting issues and often fails to learn an optimal system and signal representation (code). In addition, less is known about the autoencoder ability to design channel capacity-approaching codes, i.e., codes that maximize the input-output mutual information under a certain power constraint. The task being even more formidable for an unknown channel for which the capacity is unknown and therefore it has to be learnt. In this paper, we address the challenge of designing capacity-approaching codes by incorporating the presence of the communication channel into a novel loss function for the autoencoder training. In particular, we exploit the mutual information between the transmitted and received signals as a regularization term in the cross-entropy loss function, with the aim of controlling the amount of information stored. By jointly maximizing the mutual information and minimizing the cross-entropy, we propose a theoretical approach that a) computes an estimate of the channel capacity and b) constructs an optimal coded signal approaching it. Theoretical considerations are made on the choice of the cost function and the ability of the proposed architecture to mitigate the overfitting problem. Simulation results offer an initial evidence of the potentiality of the proposed method.

Melt pool temperature contains abundant information on metallurgical and mechanical aspects of products produced by additive manufacturing. Forecasting melt pool temperature profile during a process can help in reducing microstructural porosity and residual stresses. Although analytical and numerical models were reported, the performance of these are questionable when applied in real-time. Hence, we developed data-driven models to address this challenge, for continuous forecasting layer-wise melt pool temperature using a hybrid deep learning technique. The melt pool temperature forecasting by the proposed CNN-LSTM model is found to be better than other benchmark models in terms of accuracy and efficiency. The model results have shown that combining CNN and LSTM networks can extract the spatial and temporal information from the melt pool temperature data. Further, the proposed model results are compared with existing statistical and machine learning models. The performance measures of the proposed CNN-LSTM model indicate a greater potential for in-situ monitoring of additive manufacturing process.

In a highly competitive and liberalized energy market, where the retail of electricity is open to many potential companies, it is essential to have tools that help make decisions and guide the design of marketing strategies. In this sense, it is essential for retailers to know the behavior of their customers to correctly define their commercial strategies. One of the most commonly used methods for this is the characterization of their consumption profiles. Fortunately, for regulatory reasons, in some countries, the monthly electricity demand of each customer is openly available to any competitor. This paper explores whether this information, especially the economic sector and geographic location of a client, is useful for determining the client’s demand profile. Specifically, data on electricity demand in Spain from more than 27 million users and for a period of 3 years are analyzed. For this purpose, the electricity consumption of every client is grouped by month and normalized. The resulting demand profiles are later clustered according to different criteria. The main finding of the research is that the combined information on economic activity and location definitely enables prediction of the demand profile. Additionally, profile quality metrics are defined and obtained for the entire dataset. The resulting profiles have a mean dispersion of 10% and a confidence interval of ±17%. To clarify the use of these metrics, several examples are detailed, showing how this profile information can be used to improve the marketing decision-making process for electricity retailers.

The nonlocal self-similarity in natural image provides an effective prior for single image super-resolution (SISR), which is beneficial to contextual information capture and performance improvement, as demonstrated by conventional SISR methods. However, it is little explored to utilize this property in deep neural networks. In this paper, we propose a self-similarity prior guided (SSPG) network to incorporate self-similarity-based nonlocal operation into deep neural network for SISR. Specifically, we design a cross-scale nearest-neighbor residual (CSNNR) block via introducing cross-scale $k$ -nearest neighbors (KNN) matching into a residual block, which can be flexibly integrated into deep networks to capture long-range correlations among multi-scale and multi-level features. Meanwhile, by stacking a CSNNR block and a sequence of wide-activated residual blocks with a local skip-connection, a multi-level residual self-similarity (MRSS) module is developed to effectively employ local and nonlocal information for detail recovery. Thus, through cascading multiple MRSS modules, the proposed SSPG network performs both self-similarity-based nonlocal operation and convolution-based local operation on multi-level features to reconstruct informative features for accurate SISR. In addition, for pursuing visually pleasing results, we apply our SSPG network to the perception-oriented SISR field by following the framework of generative adversarial networks. In particular, we explore a sample-discriminating learning mechanism based on the statistical descriptions of training samples, and include it in optimization procedure to automatically tune the contributions of different samples according to their characteristics and then focus the network on creating realistic results. Extensive quantitative and qualitative evaluations on benchmark datasets illustrate the superiority of our proposed models over the state-of-the-art methods for both distortion-oriented and perception-oriented image super-resolution tasks. [ABSTRACT FROM AUTHOR]

Advances in space-based ocean surveillance systems have improved the detection of objects from high-quality remote-sensing big data. Previous studies mainly focused on finding and recognizing objects based on deep learning and statistical frameworks. Studies have not fully explored the transparent and reasonable decision-making process for the final predicted results, which is vital for civil and military applications. An explainable attention network for fine-grained ship image classification is proposed in the present study to bridge this gap. The present study seeks to increase attention to objects’ discriminative parts and explore intrinsic relationships between multiple attention parts and predicted outcomes. Several causal multi-attention maps are generated by combining the multi-head attention mechanism and structural causal model. The convolutional filters in the last layer of the network are divided into several groups, and each group is designed to express specific semantic information under supervision of the filter loss function. The results show which parts of the objects are adopted as the key factors for the network to make the final predicted outcome. In the training process, the network is designed to rapidly focus on the salient feature of objects and played a role in guiding other parts of the network to improve the explainable capability of the network without affecting the discrimination power or compromising the classification accuracy. Extensive experiments based on two public datasets show that the network is highly effective as indicated by high classification accuracy and explainable ability. [ABSTRACT FROM AUTHOR]

While various biomimetic robotic haptic sensors are often utilized to measure multifarious physical interactions to identify material properties under human exploratory procedures (EPs), traditional methods are unable to fuse well multimodal measurements under EPs. In order to solve this problem, an innovative hybrid joint group kernel sparse coding model for material recognition under EPs is proposed. First, a series of feature representations according to the characteristics of different measures are introduced. Second, we propose an innovative hybrid fusion framework based on statistical learning with kernel sparsity to effectively fuse the measurements from robotic EPs. Third, a matched optimization algorithm is constructed to deal with the innate optimization problem in the proposed framework. Finally, we develop an evaluation based on three different classification levels (coarse, medium, and fine) with the public dataset containing 184 material classes to verify the proposed method and compare with some other similar kernel methods. The evaluation results show that the accuracy of coarse, medium, and fine classification of the proposed hybrid fusion framework is 98.4%, 96.2%, and 98.4%, respectively. The framework can be extended easily to other fusion tasks with multiple measurements. [ABSTRACT FROM AUTHOR]