Search

Unforeseen machine tool component failures cause considerable losses. This study presents a new approach to unsupervised machine component condition identification. It uses test cycle data of machine components in healthy and various faulty conditions for modelling. The novelty in the approach consists of the time series representation as features, the filtering of the features for statistical significance, and the use of this feature representation to train a clustering model. The benefit in the proposed approach is its small engineering effort, the potential for automation, the small amount of data necessary for training and updating the model, and the potential to distinguish between multiple known and unknown conditions. Online measurements on machines in unknown conditions are performed to predict the component condition with the aid of the trained model. The approach was exemplarily tested and verified on different healthy and faulty states of a grinding machine axis. For the accurate classification of the component condition, different clustering algorithms were evaluated and compared. The proposed solution demonstrated encouraging results as it accurately classified the component condition. It requires little data, is straightforward to implement and update, and is able to precisely differentiate minor differences of faults in test cycle time series. ISSN:1062-0656 ISSN:1530-8065

Objective: To evaluate various machine learning algorithms in predicting peripheral vestibular dysfunction using the dataset of the center of pressure (COP) sway during foam posturography measured from patients with dizziness. Study Design: Retrospective study. Setting: Tertiary referral center. Patients: Seventy-five patients with vestibular dysfunction and 163 healthy controls were retrospectively recruited. The dataset included the velocity, the envelopment area, the power spectrum of the COP for three frequency ranges and the presence of peripheral vestibular dysfunction evaluated by caloric testing in 75 patients with vestibular dysfunction and 163 healthy controls. Main Outcome Measures: Various forms of machine learning algorithms including the Gradient Boosting Decision Tree, Bagging Classifier, and Logistic Regression were trained. Validation and comparison were performed using the area under the curve (AUC) of the receiver operating characteristic curve (ROC) and the recall of each algorithm using K-fold cross-validation. Results: The AUC (0.90 ± 0.06) and the recall (0.84 ± 0.07) of the Gradient Boosting Decision Tree were the highest among the algorithms tested, and both of them were significantly higher than those of the logistic regression (AUC: 0.85 ± 0.08, recall: 0.78 ± 0.07). The recall of the Bagging Classifier (0.82 ± 0.07) was also significantly higher than that of logistic regression. Conclusion: Machine learning algorithms can be successfully used to predict vestibular dysfunction as identified using caloric testing with the dataset of the COP sway during posturography. The multiple algorithms should be evaluated in each clinical dataset since specific algorithm does not always fit to any dataset. Optimization of the hyperparameters in each algorithm are necessary to obtain the highest accuracy.

Navigating through densely populated urban areas is one of the most important challenges for self-driving vehicles. Accurate predictions are required to enable safe and efficient interactions with other road users. Pedestrians in particular pose major problems for current state of the art prediction systems. Apart from well-understood short-term predictions, used for Automatic Emergency Braking Systems, long-term predictions remain largely unresolved. In this thesis, we aim to advance pedestrian prediction systems to enable human- understandable automated driving. In view of a vehicle-centred road infrastructure with high vehicle speeds and scarce pedestrian crossings, early detection of pedestrian intentions and movements are the key to enabling such behaviour. These detections can enable automated vehicles to perform light brake manoeuvres at an early stage in order to let a pedestrian pass. This can eliminate the need to stop, which could significantly improve traffic flow and at the same time increase overall safety. Long-term full trajectory predictions are both costly and error-prone. Therefore we propose a hierarchical prediction system that splits the prediction into multiple simplified sub-problems using domain knowledge. Each sub-problem is designed to predict a meaningful part of pedestrian movement and to detect and remove pedestrians that are irrelevant for the current scenario as early as possible. Utilizing the given road geometry to identify crosswalks we first predict the pedestrians’ hidden intent to cross the road. For all crossing pedestrians we then propose a sparse motion prediction, providing a small set of key figures instead of a full trajectory. We claim that these domain-specific key figures, namely a time-to- cross and designated crossing point, are more than sufficient to describe future pedestrian motions for the planning system of an automated vehicle. To overcome problems from over-confident single value predictions we propose to utilize Quantile Regression techniques to predict reasonable uncertainties. Our evaluations show that we are able to robustly classify the pedestrians’ hidden intent using both standard and deep learning algorithms. Additionally we show that our hierarchical prediction system, including the sparse motion prediction, is suitable for a real-time system integration. With our large real-world dataset, featuring recordings from different crosswalks and days, we provide an evaluation regarding prediction accuracy, computational load and generalizability. During this analysis, we also found indications that it might be possible to transfer trained models to previously unseen pedestrian crossings if the road geometry has at least approximately the same pavement dimensions. Furthermore, we show how our sparse motion prediction can be integrated into a situation-based planning approach to allow safe and efficient real-time interactions with other traffic participants. For this we evaluate different interaction scenarios regarding safety, time efficiency and comfort impairment. We were able to show that in most of the scenarios it is possible to minimize movement jerks and eliminate the need to stop. The overall performance is only limited by very high traffic densities.

Failures on machine tools not only occur on main components, but also on auxiliaries like cooling units or oil mist separators, which causes productivity losses similar to failures on main machine components. Due to their separation from the machine’s control network, their health status is in most cases not monitored. In this study, a new approach for online condition monitoring of auxiliary units by the example of an oil mist separator connected to a 5-axis machine tool is presented. The data is analyzed via machine learning principles in order to deduct an adequate condition assessment, encompassing environmental influences., Procedia CIRP, 88, ISSN:2212-8271, 13th CIRP Conference on Intelligent Computation in Manufacturing Engineering

Unforeseen machine tool component failures cause considerable losses. This study presents a new approach to unsupervised machine component condition identification. It uses test cycle data of machine components in healthy and various faulty conditions for modelling. The novelty in the approach consists of the time series representation as features, the filtering of the features for statistical significance, and the use of this feature representation to train a clustering model. The benefit in the proposed approach is its small engineering effort, the potential for automation, the small amount of data necessary for training and updating the model, and the potential to distinguish between multiple known and unknown conditions. Online measurements on machines in unknown conditions are performed to predict the component condition with the aid of the trained model. The approach was exemplarily tested and verified on different healthy and faulty states of a grinding machine axis. For the accurate classification of the component condition, different clustering algorithms were evaluated and compared. The proposed solution demonstrated encouraging results as it accurately classified the component condition. It requires little data, is straightforward to implement and update, and is able to precisely differentiate minor differences of faults in test cycle time series., Journal of Materials Processing & Manufacturing Science, 4 (S 3), ISSN:1062-0656, ISSN:1530-8065

We propose a novel node embedding of directed graphs to statistical manifolds, which is based on a global minimization of pairwise relative entropy and graph geodesics in a non-linear way. Each node is encoded with a probability density function over a measurable space. Furthermore, we analyze the connection between the geometrical properties of such embedding and their efficient learning procedure. Extensive experiments show that our proposed embedding is better preserving the global geodesic information of graphs, as well as outperforming existing embedding models on directed graphs in a variety of evaluation metrics, in an unsupervised setting.

Drug discovery is the process of producing compounds with desired biological properties towards diseases of interest. Considering the enormous number of potential structures one could produce, the achievements seen in modern pharmaceutical science are astounding. Despite this, the immense amount of resources required to develop a drug compound presents a bottleneck in the development of pharmaceutical compounds. Computational methods, such as machine learning, offer a potential solution to this problem. Their major advantage is that, by proposing structures in silico, they can narrow the search space, reducing the number of compounds required to be synthesized and tested. However, the use of such methods is often encumbered with issues of synthetic feasibility. To aid in this issue, we present the algorithm DINGOS (Design of Innovative NCEs Generated through Optimization Strategies). DINGOS combines the predictive capabilities of computational methods with a rule-based model of synthesizability. DINGOS is a ligand-based scoring method, generating structures which are similar to the template ligand, while within the provided definition of synthesizability. The functionality of the DINGOS algorithm was demonstrated in a case-study, in which designs were proposed for four compounds (alectinib, cariprazine, osimertinib, and pimavanserin). For each template, a set of 300 designs was proposed, out of which, above 50% were predicted as active by target prediction. Three out of four selected candidate structures were successfully synthesized with the synthetic pathway predicted by DINGOS. Of three synthesized compounds, one showed activity towards the 5-HT2B serotonin receptor. The modular nature of the DINGOS algorithm was demonstrated in a series of follow-up studies. In the first follow-up study, DINGOS was paired with a custom automated synthesis system in order to generate de novo designs within an autonomous active learning drug design cycle. Two rounds of automated synthesis were performed, generating a total of 22 novel compounds, of which five showed micromolar activity towards carbonic anhydrase II. The method was further extended, first by modifying the core predictive component of the algorithm, allowing for the internal representation of the scoring function to be flexibly changed, and in the second, DINGOS was combined with a MCTS (Monte Carlo tree search) algorithm in order to allow for non-greedy optimization of the designed structures. The results of this work showcase the potential of the DINGOS algorithm as a method for generating synthetically feasible de novo designs, and highlights DINGOS’ ability to be adapted to a variety of different drug design problems, narrowing the gap between in silico and experimental drug design.

This paper proposes a design of hierarchical fuzzy inference tree (HFIT). An HFIT produces an optimum tree-like structure, i.e., a natural hierarchical structure that accommodates simplicity by combining several low-dimensional fuzzy inference systems (FISs). Such a natural hierarchical structure provides a high degree of approximation accuracy. The construction of HFIT takes place in two phases. Firstly, a nondominated sorting based multiobjective genetic programming (MOGP) is applied to obtain a simple tree structure (a low complexity model) with a high accuracy. Secondly, the differential evolution algorithm is applied to optimize the obtained tree’s parameters. In the derived tree, each node acquires a different input’s combination, where the evolutionary process governs the input’s combination. Hence, HFIT nodes are heterogeneous in nature, which leads to a high diversity among the rules generated by the HFIT. Additionally, the HFIT provides an automatic feature selection because it uses MOGP for the tree’s structural optimization that accepts inputs only relevant to the knowledge contained in data. The HFIT was studied in the context of both type-1 and type-2 FISs, and its performance was evaluated through six application problems. Moreover, the proposed multiobjective HFIT was compared both theoretically and empirically with recently proposed FISs methods from the literature, such as McIT2FIS, TSCIT2FNN, SIT2FNN, RIT2FNS-WB, eT2FIS, MRIT2NFS, IT2FNN-SVR, etc. From the obtained results, it was found that the HFIT provided less complex and highly accurate models compared to the models produced by the most of other methods. Hence, the proposed HFIT is an efficient and competitive alternative to the other FISs for function approximation and feature selection., IEEE Transactions on Fuzzy Systems, 26 (2), ISSN:1063-6706, ISSN:1941-0034

The need to cope with the climate change challenge has given a strong impulse to promote energy produced by Renewable Energy Sources (RES), phasing out large plants based on fossil fuels in the transmission networks. Some of the most notable developments foreseen in power systems target Distribution Networks (DNs). In the future, DNs are expected to host a large percentage of distributed generators, including RES, to supply a growing share of the total demand. These units, in combination with other Distributed Energy Resources (DERs) such as electric vehicles, Battery Energy Storage Systems (BESSs) and Controllable Loads (CLs), are gradually elevating the role of Distribution System Operators (DSOs), allowing them to provide ancillary services and support the bulk transmission system. However, this new paradigm introduces significant challenges to the DNs which were not designed to host generation, and are currently operating close to their physical limits. They face increased uncertainties due to intermittent RES generation and new consumption patterns which make the consumer response hard to predict. Finally, bi-directional flows may trigger power quality issues, e.g. overvoltages, and the need to revise the protection schemes. The main scope of this dissertation is to develop methods to exploit the operational DER flexibility in every time frame of distribution systems in the most cost-effective way. Emphasis is put on optimal centralized approaches assuming perfect communication and monitoring infrastructure, and on robust and scalable purely local control strategies. Furthermore, the thesis investigates ancillary service provision to higher voltage levels and reliability aspects based on protection and automation configurations. As active measures this dissertation considers Distributed Generator (DG) reactive power control and active power curtailment, BESSs, CLs and on-load-tap-changing transformers. More specifically, the first part of the dissertation proposes various formulations of a tractable Optimal Power Flow (OPF) model based on the iterative Backward/Forward Sweep (BFS) power flow. The BFS-OPF is tractable due to the iterative scheme nature and the efficient power flow calculation, guarantees AC feasible solutions, and is used as the core tool in the operation-aware grid planning stage, the operational planning stage under uncertainty, and the real-time operation of active DNs. The second part proposes and verifies experimentally data-driven methods to emulate the optimal response without the need for communication considering stability analysis of the closed-loop control strategies. Segmented and multiple linear regression, support vector machines for the regression and classification problems, as well as hierarchical and partition clustering approaches are used to derive the data-driven schemes and tackle scalability aspects. Finally, the third part develops OPF-based schemes to offer frequency control regulation considering islanding constraints and voltage support according to European market frameworks, as well as identify optimal protection and automation configurations by a brute-force and a genetic algorithm method. This dissertation shows how advanced planning, operation, and control schemes in modern DNs can be used to tackle the challenges posed by the low-carbon future transformation efforts. Active measures can defer costly investments, solve power quality issues, and unlock new business cases for DSOs. Reactive power control is recognized to have the largest flexibility potential in DNs, since it can minimize losses, mitigate voltage issues, tackle unbalances in multi-phase systems and provide voltage support to higher voltage level, comprising the cheapest active measure., ISBN:978-3-906916-75-0

Over the past two decades, the feedforward neural network (FNN) optimization has been a key interest among the researchers and practitioners of multiple disciplines. The FNN optimization is often viewed from the various perspectives: the optimization of weights, network architecture, activation nodes, learning parameters, learning environment, etc. Researchers adopted such different viewpoints mainly to improve the FNN's generalization ability. The gradient-descent algorithm such as backpropagation has been widely applied to optimize the FNNs. Its success is evident from the FNN's application to numerous real-world problems. However, due to the limitations of the gradient-based optimization methods, the metaheuristic algorithms including the evolutionary algorithms, swarm intelligence, etc., are still being widely explored by the researchers aiming to obtain generalized FNN for a given problem. This article attempts to summarize a broad spectrum of FNN optimization methodologies including conventional and metaheuristic approaches. This article also tries to connect various research directions emerged out of the FNN optimization practices, such as evolving neural network (NN), cooperative coevolution NN, complex-valued NN, deep learning, extreme learning machine, quantum NN, etc. Additionally, it provides interesting research challenges for future research to cope-up with the present information processing era., Engineering Applications of Artificial Intelligence, 60, ISSN:0952-1976

Intelligent tutoring systems (ITS) are gaining importance in education, as they can individualize the training for each learner. The individualization relies on the student model that infers cognitive, metacognitive or affective states of the learner based on the history of interactions with the system. Research on student models has mainly focused on how to better model student knowledge and student learning, but recently, models of metacognitive and affective states, as well as automatic assessments of students, has gained interest. In this thesis, we present data-driven models for the identification and discovery of student traits. We focus on sensor-free models relying entirely on interaction logs. First, we present a completely supervised pipeline for integrating automatic assessment seamlessly into an ITS and apply the method to the case of developmental dyscalculia (DD). We demonstrate that interaction logs provide enough information to identify children at risk of DD with high accuracy. In addition, we demonstrate test validity and reliability comparable to traditional assessments. Our model is able to adapt the duration of the assessment to the individual child and can classify a child at risk of DD with an accuracy of 91% after 11 minutes on average. Second, we present a semi-supervised classification pipeline that makes effective use of unlabeled data to significantly improve model quality for detecting student characteristics. We employ deep variational auto-encoders to learn efficient feature embeddings that improve the performance of standard classifiers by up to 28%. We demonstrate that our method outperforms previous methods for finding efficient feature embeddings and generalizes better to imbalanced data sets compared to expert features. The method provides the ability to improve performance on a variety of classification tasks in educational data mining. Third, we propose an unsupervised evolutionary clustering pipeline to discover student characteristics. Our pipeline can be used as a black box for any ITS and is optimized to improve cluster stability over multiple training sessions in the presence of noise. Our model selection is designed such that relevant cluster evolution effects can be captured. Our method outperforms previous work regarding clustering performance and stability on synthetic data. We demonstrate that the clustering pipeline is able to detect interesting student behavior and properties of learning environments based on data from real-world ITS.

2015 IEEE International Conference on Computer Vision Workshop (ICCVW), ISBN:978-1-4673-9711-7

Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '16), ISBN:978-1-4503-4462-3

A recent trend in condensed matter physics is that of taking systems out-of-equilibrium. In such a setting, physical phenomena are possible that do not have equilibrium counterparts. Two typical scenarios for non-equilibrium physics are that of open systems, i.e. systems that couple to an environment, and that of quenches, where the system is suddenly taken out-of-equilibrium and its relaxation is studied. The present dissertation addresses three questions, two of which deal with these two scenarios of non-equilibrium situations. The third question is slightly orthogonal, and asks about the potential applications of machine learning to condensed matter physics. As the first question, we asked about the fate of one dimensional symmetry protected topological (SPT) states when coupled to an environment. By showing how to extend the concept of entanglement spectra for pure states to that of mixed states, we were able to identify genuine mixed states with the properties of topologically non-trivial states. Additionally, we showed how to extend the classification scheme of SPT states based on projective symmetry representations to open systems. As an example, we have also considered the evolution of initial states that are topologically non-trivial upon coupling them to an environment. We have found that the topological non-trivial nature of the state dissapears on the way to reaching a steady state. As the second question, we asked about the relaxation to equilibrium of a single spin probe coupled to an environment. We showed that the transient behaviour of the de-coherence of this probe spin can be used to investigate an interacting and disordered spin chain. For weak disorder this spin chain is in a thermalizing phase, whereas for strong disorder the system avoids thermalization and instead forms a many-body localized (MBL) phase. We showed that the de-coherence properties of the spin probe provide an unambiguous signal of the MBL phase. This work provides an important experimental probe for investigating the properties of many-body quantum systems further. For the third question, we considered an equilibrium scenario and asked what potential use a machine learning algorithm can be for such cases. We showed that by compressing the information of the wave function in the form of the entanglement spectrum, it is possible to detect phase transitions using an approach based on artificial neural networks. The application of machine learning techniques to condensed matter physics is a very novel trend, and it is a fast-moving subject. Next to developing our own approach in this dissertation, we provide an overview of the recent works, thereby giving an understanding of the potential uses of machine learning in condensed matter physics. These three projects together, each in their own right, address important questions in condensed matter physics. Each of them deals with a recent topic that is of interest to a broad community, and provides non-trivial results that further these communities.

Adaptive learning introduces persistence in the evolution of agents’ beliefs over time. For applied purposes this is a convenient feature to help explain why economies present sluggish adjustments towards equilibrium. The pace of learning is directly determined by the gain parameter, which regulates how quickly new information is incorporated into agents’ beliefs. We document renewed empirical calibrations of plausible gain values for adaptive learning applications to macroeconomic data. We cover a broad range of model speci- fications of applied interest. Our analysis also includes innovative approaches to the en- dogenous determination of time-varying gains in real-time, and a thorough discussion of the different theoretical interpretations of the learning gain. We also evaluate the merits of different approaches to the gain calibration according to their performance in forecasting macroeconomic variables and in matching survey forecasts. Our results indicate a great degree of heterogeneity in the gain calibrations according to the variable forecasted and the lag length of the model specifications. Calibrations to match survey forecasts are found to be lower than those derived according to the forecast- ing performance, suggesting some degree of bounded rationality in the speed with which agents update their beliefs., KOF Working Papers, 392

The use of machine learning techniques has become increasingly widespread in commercial applications and academic research. Machine learning algorithms learn a model from data that allows computers to make and improve predictions or behaviors. Despite their popularity and usefulness, most machine learning techniques require expert knowledge to guide the decisions about the most appropriate model and settings for a particular problem. In many cases, expert knowledge is not readily available. When it is, the complexity of the problem and subjectivity of the expert can often lead to sub-optimal choices in the machine learning strategy. Since different machine learning techniques are suitable for different problems, choosing the right technique and fine-tuning its particular settings are crucial tasks that will directly impact the quality of the predictions. However, deciding which machine learning technique is most well suited for processing specific data is not an easy task, as the number of choices is usually very large. In this work, we present a method that automatically selects the best machine learning algorithm for a particular set of data, and optimizes its parameter settings. Our approach is flexible and customizable, enabling the user to specify their needs in terms of predictive power, sensitivity, specificity, consistency of the predictions, and speed, among other criteria. The results obtained show that using the machine learning technique and configuration sug- gested by our automated approach yields predictions of a much higher quality than selecting the technique with the best results under its default settings. We also present a method to efficiently guide the search for optimal parameter settings by identifying ranges of values for each setting that produce good results for most problems. By transferring this knowledge to new problems, it is possible to find the optimal configuration of the algorithm more quickly.

We compare forecasts from different adaptive learning algorithms and calibrations ap- plied to US real-time data on inflation and growth. We find that the Least Squares with constant gains adjusted to match (past) survey forecasts provides the best overall perfor- mance both in terms of forecasting accuracy and in matching (future) survey forecasts., KOF Working Papers, 356