Search

Glioma is the most common type of tumor in humans originating in the brain. According to the World Health Organization, gliomas can be graded on a four-stage scale, ranging from the most benign to the most malignant. The grading of these tumors from image information is a far from trivial task for radiologists and one in which they could be assisted by machine-learning-based decision support. However, the machine learning analytical pipeline is also fraught with perils stemming from different sources, such as inadvertent data leakage, adequacy of 2D image sampling, or classifier assessment biases. In this paper, we analyze a glioma database sourced from multiple datasets using a simple classifier, aiming to obtain a reliable tumor grading and, on the way, we provide a few guidelines to ensure such reliability. Our results reveal that by focusing on the tumor region of interest and using data augmentation techniques we significantly enhanced the accuracy and confidence in tumor classifications. Evaluation on an independent test set resulted in an AUC-ROC of 0.932 in the discrimination of low-grade gliomas from high-grade gliomas, and an AUC-ROC of 0.893 in the classification of grades 2, 3, and 4. The study also highlights the importance of providing, beyond generic classification performance, measures of how reliable and trustworthy the model’s output is, thus assessing the model’s certainty and robustness. Carla Pitarch is a fellow of Eurecat’s “Vicente López” PhD grant program.

Glioma is the most common type of tumor in humans originating in the brain. According to the World Health Organization, gliomas can be graded on a four-stage scale, ranging from the most benign to the most malignant. The grading of these tumors from image information is a far from trivial task for radiologists and one in which they could be assisted by machine-learning-based decision support. However, the machine learning analytical pipeline is also fraught with perils stemming from different sources, such as inadvertent data leakage, adequacy of 2D image sampling, or classifier assessment biases. In this paper, we analyze a glioma database sourced from multiple datasets using a simple classifier, aiming to obtain a reliable tumor grading and, on the way, we provide a few guidelines to ensure such reliability. Our results reveal that by focusing on the tumor region of interest and using data augmentation techniques we significantly enhanced the accuracy and confidence in tumor classifications. Evaluation on an independent test set resulted in an AUC-ROC of 0.932 in the discrimination of low-grade gliomas from high-grade gliomas, and an AUC-ROC of 0.893 in the classification of grades 2, 3, and 4. The study also highlights the importance of providing, beyond generic classification performance, measures of how reliable and trustworthy the model’s output is, thus assessing the model’s certainty and robustness.

In January 2021, the DOD released its first Counter-Small Unmanned Aircraft Systems Strategy to address the growing risk to military personnel, facilities, and assets posed by the rapid technological advancement and proliferation of sUAS. Existing counter-drone capabilities—heavily reliant on electronic warfare to disrupt the communication link between user and device—no longer address an evolving threat that includes autonomous drones, COTS technology, and an increasing number of drones in the airspace that can overwhelm a C-sUAS operator. To counter the increasingly complex small drone threat, the Army-led Joint Counter-sUAS Office is pursuing materiel and non-materiel solutions for its new system-of-systems approach. One vexing C-sUAS challenge involves radar detection systems discriminating some sUAS from other flying objects, like birds, due to their comparable size, slow movement, and low altitude. Inaccurate or inefficient sUAS classification using radar data can be a force protection threat due to the limited number of electro-optical sensors and human operators for classification at-scale. This thesis uses bird and drone radar track data from two different training environments to explore hidden structure in the data, develop independent unsupervised and supervised learning models using the two datasets, and experiment with data sampling and feature engineering to improve upon model robustness to different environments and dynamic environmental conditions. Lieutenant Colonel, United States Army Approved for public release. Distribution is unlimited.

In this work, we formalize the concept of differential model robustness (DMR), a new property for ensuring model security in federated learning (FL) systems. For most conventional FL frameworks, all clients receive the same global model. If there exists a Byzantine client who maliciously generates adversarial samples against the global model, the attack will be immediately transferred to all other benign clients. To address the attack transferability concern and improve the DMR of FL systems, we propose the notion of differential model distribution (DMD) where the server distributes different models to different clients. As a concrete instantiation of DMD, we propose the ARMOR framework utilizing differential adversarial training to prevent a corrupted client from launching white-box adversarial attack against other clients, for the local model received by the corrupted client is different from that of benign clients. Through extensive experiments, we demonstrate that ARMOR can significantly reduce both the attack success rate (ASR) and average adversarial transfer rate (AATR) across different FL settings. For instance, for a 35-client FL system, the ASR and AATR can be reduced by as much as 85% and 80% over the MNIST dataset.

Brain aneurysm detection in Time-Of-Flight Magnetic Resonance Angiography (TOF-MRA) has undergone drastic improvements with the advent of Deep Learning (DL). However, performances of supervised DL models heavily rely on the quantity of labeled samples, which are extremely costly to obtain. Here, we present a DL model for aneurysm detection that overcomes the issue with ''weak'' labels: oversized annotations which are considerably faster to create. Our weak labels resulted to be four times faster to generate than their voxel-wise counterparts. In addition, our model leverages prior anatomical knowledge by focusing only on plausible locations for aneurysm occurrence. We frst train and evaluate our model through cross-validation on an in-house TOF-MRA dataset comprising 284 subjects (170 females / 127 healthy controls / 157 patients with 198 aneurysms). On this dataset, our best model achieved a sensitivity of 83%, with False Positive (FP) rate of 0.8 per patient. To assess model generalizability, we then participated in a challenge for aneurysm detection with TOF-MRA data (93 patients, 20 controls, 125 aneurysms). On the public challenge, sensitivity was 68% (FP rate=2.5), ranking 4th/18 on the open leaderboard. We found no signifcant diference in sensitivity between aneurysm risk-of-rupture groups (p=0.75), locations (p=0.72), or sizes (p=0.15). Data, code and model weights are released under permissive licenses. We demonstrate that weak labels and anatomical knowledge can alleviate the necessity for prohibitively expensive voxel-wise annotations., Paper accepted as Original Article in the journal Neuroinformatics (https://link.springer.com/article/10.1007/s12021-022-09597-0)

Klasifikacija zvuka postaje sve popularniji zadatak s različitim primjenama u današnjem svijetu. U ovom radu smo se bavili klasifikacijom zvuka pomoću modela rezidualne neuronske mreže gdje su ulazni podaci bili spektrogrami dobiveni iz tri transformacije od interesa: Fourierove transformacije na vremenskom otvoru, mel-frekvencijskih kepstralnih koeficijenata i diskretne valićne transformacije. U prvom dijelu rada postavili smo teorijske osnove obrade zvuka pomoću ovih transformacija prije nego što smo prešli na duboko rezidualno učenje i protivničke napade kojima je testirana robusnost modela. Na kraju smo na praktičnom primjeru pokazali treniranje i validaciju jednog rezidualnog modela te komentirali rezultate iz prijašnjih istraživanja uz numerički prikaz rješenja. Rad smo zaključili s komentarima o svim konceptima kroz koje smo prošli te se dotaknuli potencijalnog budućeg istraživanja. Sound classification is an increasingly popular task with different applications in today's world. In this paper we've talked about sound classification using a residual neural network model where the input data were spectrograms obtained from three transformations of interest: short-time Fourier transform, mel-frequency cepstral coefficients and discrete wavelet transform. In the first half of the paper we've set a theoretical basis of sound processing via these transformations before delving into the definition of deep residual learning and adversarial attacks which were used to test the model robustness. In the end we've shown a practical example of training and validating of one such residual model, while commenting the results of prior research with numeric data displayed. We've concluded the paper with comments about all traversed concepts and touched upon potential future research.

Structural equation models (SEM) are widely used in the social sciences. They model the relationships between latent variables in structural models, while defining the latent variables by observed variables in measurement models. Frequently, it is of interest to compare particular parameters in an SEM as a function of a discrete grouping variable. Multiple-group SEM is employed to compare structural relationships between groups. In this article, estimation approaches for the multiple-group are reviewed. We focus on comparing different estimation strategies in the presence of local model misspecifications (i.e., model errors). In detail, maximum likelihood and weighted least-squares estimation approaches are compared with a newly proposed robust Lp loss function and regularized maximum likelihood estimation. The latter methods are referred to as model-robust estimators because they show some resistance to model errors. In particular, we focus on the performance of the different estimators in the presence of unmodelled residual error correlations and measurement noninvariance (i.e., group-specific item intercepts). The performance of the different estimators is compared in two simulation studies and an empirical example. It turned out that the robust loss function approach is computationally much less demanding than regularized maximum likelihood estimation but resulted in similar statistical performance.

The modeling and compensation method is a common method for reducing the influence of thermal error on the accuracy of machine tools. The prediction accuracy and robustness of the thermal error model are two key performance measures for evaluating the compensation effect. However, it is difficult to maintain the prediction accuracy and robustness at the desired level when the ambient temperature exhibits strong seasonal variations. Therefore, a year-round thermal error modeling and compensation method for the spindle of machine tools based on ambient temperature intervals (ATIs) is proposed in this paper. First, the ATIs applicable to the thermal error prediction models (TEPMs) under different ambient temperatures are investigated, where the C-Means clustering algorithm is utilized to determine ATIs. Furthermore, the prediction effect of different numbers of ATIs is analyzed to obtain the optimal number of ATIs. Then, the TEPMs corresponding to different ATIs in the annual ambient temperature range are established. Finally, the established TEPMs of ATIs are used to predict the experimental data of the entire year, and the prediction accuracy and robustness of the proposed ATI model are analyzed and compared with those of the low and high ambient temperature models. The prediction accuracies of the ATI model are 20.6% and 41.7% higher than those of the low and high ambient temperature models, respectively, and the robustness is improved by 48.8% and 62.0%, respectively. This indicates that the proposed ATI method can achieve high prediction accuracy and robustness regardless of the seasonal temperature variations throughout the year.

Resistive random access memory (RRAM) is a promising technology for energy-efficient neuromorphic accelerators. However, when a pretrained deep neural network (DNN) model is programmed to an RRAM array for inference, the model suffers from accuracy degradation due to RRAM nonidealities, such as device variations, quantization error, and stuck-at-faults. Previous solutions involving multiple read–verify–write (R-V-W) to the RRAM cells require cell-by-cell compensation and, thus, an excessive amount of processing time. In this article, we propose a joint algorithm-design solution to mitigate the accuracy degradation. We first leverage knowledge distillation (KD), where the model is trained with the RRAM nonidealities to increase the robustness of the model under device variations. Furthermore, we propose random sparse adaptation (RSA), which integrates a small on-chip memory with the main RRAM array for postmapping adaptation. Only the on-chip memory is updated to recover the inference accuracy. The joint algorithm-design solution achieves the state-of-the-art accuracy of 99.41% for MNIST (LeNet-5) and 91.86% for CIFAR-10 (VGG-16) with up to 5% parameters as overhead while providing a 15– $150\times $ speedup compared with R-V-W.

An important instrument for achieving smart and high-performance buildings is Machine Learning (ML). A lot of research has been done in exploring the ML models for various applications in the built environment such as occupancy prediction. Nevertheless, the research focused mostly on analyzing the feasibility and performance of different supervised ML models but has rarely focused on practical applications and the scalability of those models. In this study, a transfer learning method is proposed as a solution to typical problems in the practical application of ML in buildings. Such problems are scaling a model to a different building, collecting ground truth data necessary for training the supervised model, and assuring the model is robust when conditions change. The practical application examined in this work is a deep learning model used for predicting room occupancy using indoor climate IoT sensors. This work proved that it is possible to significantly reduce the length of ground truth data collection to only two days. The robustness of the transferred model was tested as well, where performance stayed on a similar level if a suitable normalization technique was used. In addition, the proposed methodology was tested with room occupancy level prediction, showing slightly lower performance. Finally, the importance of understanding the performance metrics is crucial for market adoption of ML-based solutions in the built environment. Therefore, in this study, additional analysis was done by presenting the occupancy prediction model performance in understandable ways from the practical perspective.

Funding Information: Funding: This research was funded by Business Finland through the project Mad@Work. Funding Information: This research was funded by Business Finland through the project Mad@Work. The authors would like to thank Aalto University and Granlund Oy for making this research possible. Publisher Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland. An important instrument for achieving smart and high-performance buildings is Machine Learning (ML). A lot of research has been done in exploring the ML models for various applications in the built environment such as occupancy prediction. Nevertheless, the research focused mostly on analyzing the feasibility and performance of different supervised ML models but has rarely focused on practical applications and the scalability of those models. In this study, a transfer learning method is proposed as a solution to typical problems in the practical application of ML in buildings. Such problems are scaling a model to a different building, collecting ground truth data necessary for training the supervised model, and assuring the model is robust when conditions change. The practical application examined in this work is a deep learning model used for predicting room occupancy using indoor climate IoT sensors. This work proved that it is possible to significantly reduce the length of ground truth data collection to only two days. The robustness of the transferred model was tested as well, where performance stayed on a similar level if a suitable normalization technique was used. In addition, the proposed methodology was tested with room occupancy level prediction, showing slightly lower performance. Finally, the importance of understanding the performance metrics is crucial for market adoption of ML-based solutions in the built environment. Therefore, in this study, additional analysis was done by presenting the occupancy prediction model performance in understandable ways from the practical perspective.

The Portable Document Format (PDF) is considered one of the most popular formats due to its flexibility and portability across platforms. Although people have used machine learning techniques to detect malware in PDF files, the problem with these models is their weak resistance against evasion attacks, which constitutes a major security threat. The goal of this study is to introduce three machine learning-based systems that enhance malware detection in the presence of evasion attacks by substantially relying on evasive data to train malware and evasion detection models. To evaluate the robustness of the proposed systems, we used two testing datasets, a real dataset containing around 100,000 PDF samples and an evasive dataset containing 500,000 samples that we generated. We compared the results of the proposed systems to a baseline model that was not adversarially trained. When tested against the evasive dataset, the proposed systems provided an increase of around 80% in the f1-score compared to the baseline. This proves the value of the proposed approaches towards the ability to deal with evasive attacks.

Named entity recognition is an important basic task in the field of natural language processing. The current mainstream named entity recognition methods are mainly based on the deep neural network model. The vulnerability of the deep neural network itself leads to a significant decline in the accuracy of named entity recognition when there is adversarial text in the text. In order to improve the robustness of named entity recognition under adversarial conditions, this paper proposes a Chinese named entity recognition model based on fusion graph embedding. Firstly, the model encodes and represents the phonetic and glyph information of the input text through graph learning and integrates above-multimodal knowledge into the model, thus enhancing the robustness of the model. Secondly, we use the Bi-LSTM to further obtain the context information of the text. Finally, conditional random field is used to decode and label entities. The experimental results on OntoNotes4.0, MSRA, Weibo, and Resume datasets show that the F1 values of this model increased by 3.76%, 3.93%, 4.16%, and 6.49%, respectively, in the presence of adversarial text, which verifies the effectiveness of this model.

This research aims to compare the performance of ARIMA as a linear model with that of the combination of ARIMA and GARCH family models to forecast S&P500 log returns in order to construct algorithmic investment strategies on this index. We used the data collected from Yahoo Finance with daily frequency for the period from 1 January 2000 to 31 December 2019. By using a rolling window approach, we compared ARIMA with the hybrid models to examine whether hybrid ARIMA-SGARCH and ARIMA-EGARCH can really reflect the specific time-series characteristics and have better predictive power than the simple ARIMA model. In order to assess the precision and quality of these models in forecasting, we compared their equity lines, their forecasting error metrics (MAE, MAPE, RMSE, MAPE), and their performance metrics (annualized return compounded, annualized standard deviation, maximum drawdown, information ratio, and adjusted information ratio). The main contribution of this research is to show that the hybrid models outperform ARIMA and the benchmark (Buy&Hold strategy on S&P500 index) over the long term. These results are not sensitive to varying window sizes, the type of distribution, and the type of the GARCH model.

An automatic electrocardiogram (ECG) myocardial infarction detection system needs to satisfy several requirements to be efficient in real-world practice. These requirements, such as reliability, less complexity, and high performance in decision-making, remain very important in a realistic clinical environment. In this study, we investigated an automatic ECG myocardial infarction detection system and presented a new approach to evaluate its robustness and durability performance in classifying the myocardial infarction (with no feature extraction) under different noise types. We employed three well-known supervised machine learning models: support vector machine (SVM), k-nearest neighbors (KNN), and random forest (RF), and tested the performance and robustness of these techniques in classifying normal (NOR) and myocardial infarction (MI) using real ECG records from the PTB database after normalization and segmentation of the data, with a suggested inter-patient paradigm separation as well as noise from the MIT-BIH noise stress test database (NSTDB). Finally, we measured four metrics: accuracy, precision, recall, and F1-score. The simulation revealed that all of the models performed well, with values of over 0.50 at lower SNR levels, in terms of all the metrics investigated against different types of noise, indicating that they are encouraging and acceptable under extreme noise situations are are thus considered sustainable and robust models for specific forms of noise. All of the methods tested could be used as ECG myocardial infarction detection tools in real-world practice under challenging circumstances.

Visible-near infrared (vis–NIR) spectroscopy has been widely used to characterize soil information from field to global scales. Before applying a calibrated spectral predictive model to acquire soil information, either independent validation ork-fold cross validation is used to evaluate model performance. However, there is no consensus on which validation strategy is more suitable and robust when evaluating model performance for the studies in different scales. The objective of this study is to evaluate and compare the model performance of two validation strategies coupling different calibration sizes (a ratio of calibration to validation of 2:1, 4:1 and 9:1) and calibration sampling strategies (random sampling (RS), rank, Kennard-Stone (KS), rank-Kennard-Stone (RKS) and conditioned Latin hypercube sampling (cLHS)) across scales. A total of 17,272 vis–NIR spectra of mineral soils from LUCAS data (continental scale) and theirsoil organic carbon(SOC) and clay contents were used in this study, and the dataset was further split into national (2761 samples in France) and five regional datasets (110 to 248 samples from five French administrative regions). To eliminate the effect of changing validation set on the model performance, a consistent test set (20% of total samples at each scale) was split to evaluate all the combinations involved in two validation strategies. The Lin’s concordancecorrelation coefficient(CCC) of the cubist model were stable for both SOC and clay for different calibration sizes, calibration sampling and validation strategies for a large calibration size (>1400) at the national and continental scales. A larger calibration size can potentially improve model performance for a small dataset (k-fold cross validation would be appropriate; (2) for a small data set (dozens to hundreds),k-fold cross validation and/or repeated random sampling for independent validation would be more robust for spectral predictive model evaluation.

Many defenses have emerged with the development of adversarial attacks. Models must be objectively evaluated accordingly. This paper systematically tackles this concern by proposing a new parameter-free benchmark we coin RoBIC. RoBIC fairly evaluates the robustness of image classifiers using a new half-distortion measure. It gauges the robustness of the network against white and black box attacks, independently of its accuracy. RoBIC is faster than the other available benchmarks. We present the significant differences in the robustness of 16 recent models as assessed by RoBIC., 4 pages, accepted to ICIP 2021

Internship Report presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics Label noise is a common issue in real-life applications of machine learning for fraud detection, that can lead to sub-optimal decisions during the model building phase, and, ultimately, lead to poor model performance. A key factor to the impact of noisy data on the performance of a model is the algorithm used to train and its robustness to label noise. In this work,we studied the robustness of the models generated by two different supervised tree-based algorithms, Random Forest and LightGBM, to different types of random and not at random artificial label noise injection techniques, at different percentages of noise, and using different datasets to both train and evaluate them. We also observed the impacts of label noise in the evaluation of the performance of a model. Finally, we analyzed the importance of the different hyperparameters of both algorithms in their performance.We show that both algorithms are robust to random label noise at different noise percentages, however they fail to separate between the classes when in the presence of noise not at random. We also show that, for random label noise, the correlation between the model performance over the noisy validation set and the test set decreases as we increase the noise percentage, however, for noise not at random there is no obvious correlation between the two sets. Finally, we conclude which hyperparameters are the most relevant for the performance of Random Forest models in the presence of random label noise, and in most cases, neither of the studied hyperparameters for LightGBM seem to be more relevant than the others for model performance. Um problema comum na aplicação de técnicas de aprendizagem automática para a deteção de fraude é a rotulagem incorreta das instâncias, que pode levar a decisões sub-ótimas durante a fase de construção do modelo, e assim levar a que o mesmo tenha baixo desempenho. Um fator-chave do impacto que a rotulagem incorreta tem no desempenho de um modelo é o algoritmo usado na sua construção e o quão robusto é. Neste trabalho, estudámos a robustez de modelos gerados através de dois tipos diferentes de algoritmos de aprendizagem supervisionado baseados em árvores de decisão, Random Forest e LightGBM, a diferentes tipos de métodos de injeção de ruído, uns aleatórios e outros determinísticos. Avaliámos os resultados adicionando diferentes percentagens de perturbação no treino e na validação e analisámos o impacto do ruído tanto no treino, como na avaliação do desempenho do modelo. Por fim, analisámos a importância dos diferentes hiper-parâmetros têm para o aumento do nível de desempenho do modelo. Os nossos resultados mostram que ambos os algoritmos são robustos a diferentes percentagens de rótulos incorretos, quando estes são introduzidos de forma aleatória, contudo os algoritmos não conseguem distinguir entre casos de fraude e de não fraude quando são usados métodos determinísticos. Vamos também mostrar que, para rótulos incorretos introduzidos de forma aleatória, a correlação entre o desempenho de um modelo nos dados de validação com ruído e o desempenho do modelo nos dados de teste sem ruído, diminui à medida que aumentamos a percentagem de rótulos incorretos. Porém, para métodos determinísticos de inserção de rótulos incorretos, não se verifica nenhuma correlação entre os conjuntos de dados. Concluímos quais os hiper-parâmetros que são mais relevantes para o desempenho dos modelos de Random Forest quando consideramos a inserção aleatória de rótulos incorretos, e que para LightGBM, na maior parte das vezes, nenhum dos hiper-parâmetros estudados se parece destacar quando consideramos o desempenho do modelo.

Deep learning architectures like Convolutional Neural Networks (CNNs) are quickly becoming the standard for detecting and counting objects in digital images. However, most of the experiments found in the literature train and test the neural networks using data from a single image source, making it difficult to infer how the trained models would perform under a more diverse context. The objective of this study was to assess the robustness of models trained using data from a varying number of sources. Nine different devices were used to acquire images of yellow sticky traps containing psyllids and a wide variety of other objects, with each model being trained and tested using different data combinations. The results from the experiments were used to draw several conclusions about how the training process should be conducted and how the robustness of the trained models is influenced by data quantity and variety. Made available in DSpace on 2020-10-07T09:14:58Z (GMT). No. of bitstreams: 1 AP-CNN-psyllids-2020.pdf: 4603738 bytes, checksum: 1d3b2dd96f24da3dd75216169a2402f6 (MD5) Previous issue date: 2020

The reasonable pricing of options can effectively help investors avoid risks and obtain benefits, which plays a very important role in the stability of the financial market. The traditional single option pricing model often fails to meet the ideal expectations due to its limited conditions. Combining an economic model with a deep learning model to establish a hybrid model provides a new method to improve the prediction accuracy of the pricing model. This includes the usage of real historical data of about 10,000 sets of CSI 300 ETF options from January to December 2020 for experimental analysis. Aiming at the prediction problem of CSI 300ETF option pricing, based on the importance of random forest features, the Convolutional Neural Network and Long Short-Term Memory model (CNN-LSTM) in deep learning is combined with a typical stochastic volatility Heston model and stochastic interests CIR model in parameter models. The dual hybrid pricing model of the call option and the put option of CSI 300ETF is established. The dual-hybrid model and the reference model are integrated with ridge regression to further improve the forecasting effect. The results show that the dual-hybrid pricing model proposed in this paper has high accuracy, and the prediction accuracy is tens to hundreds of times higher than the reference model; moreover, MSE can be as low as 0.0003. The article provides an alternative method for the pricing of financial derivatives.

The present work reports on the study and the comparison of the performance of three "Genie rural" (GR) and two Water Balance (WB) models. The calibration and robustness performances are analysed in the light of the hydro-climatic conditions. The study shows that the GR models are much more efficient and robust than the WB models. The behaviour of calibration performance as a function of hydroclimatic variables varies according to the model and goodness-of-fit criteria (GOFC). The GR models are more robust in terms of NSE(Q) and NSE( root Q) and the WB models in terms of KGE. For more robustness of the models, it is better to transfer the parameters to wetter periods or periods with a lower Potential EvapoTranspiration (PET) than the calibration period. For a loss of robustness of less than 20% for GR and 30% for WB, the variation between calibration and rain validation/PET periods must be around +/- 15%/+/- 1.5%.

The diagnosis of loss of coolant accidents in nuclear reactors has attracted a great deal of attention in condition monitoring of nuclear power plants given that the health of the cooling system is crucial to the nuclear reactor's stable operation. Many different types of neural networks have commonly been applied to loss of coolant accident diagnosis. It is important to select a suitable architecture for the neural network that delivers robust results, in that the predicted break size is deemed to be accurate even for break sizes that are not included in the training data sets. The robustness metric proposed in our previous work is applied to compare the robustness of different diagnostic models. The data used for training these models consists of a number of time-series data sets, each for a different break size, with the transient behavior of different measurable variables in the coolant system of a nuclear reactor, following a simulated loss of coolant accident in a high-fidelity simulator. Given the simulation data for different break sizes, four different neural network architectures are investigated and their properties are compared and discussed. These models include a fully-connected multilayer perceptron with one hidden layer, a multilayer perceptron with one hidden layer that is pruned using the optimal brain surgeon algorithm, a fully-connected multilayer perceptron with two hidden layers, and a group method of data handling neural network. In this paper, an interpolation pre-processing method is investigated and shown to be effective to further improve the capability of neural networks for robustly predicting the break size of a loss of coolant accident. Both linear interpolation and cubic spline interpolation are studied as alternatives for the pre-processing approach. The performance of models developed with and without interpolation pre-processing are compared with the previously proposed robustness metric. Moreover, three blind cases are introduced to evaluate and compare the performance of the diagnostic models. Finally, a combined diagnostic model is proposed based on three different architectures to obtain high prediction accuracy and good robustness.

According to literature bankruptcy prediction models are less accurate if applied in under alternative conditions. In our previous research we created our own bankruptcy prediction model. When creating the model we tried to applicate an approach different to previous ones. For creating the model we used the traditional method of linear discrimination analysis, but we employed only transformed variables with approximately normal distribution. What is more, the variables pairs are mostly negatively correlated. According to literature such factors should positively influence the model accuracy. However there is a very limited literacy how such application affects the stability of model’s accuracy. The aim of this paper is to analyse the stability of model’s accuracy in application in different time period or different line of business. Moreover, we aim to examine and discuss the effectiveness of the procedure which was used to create the model. Podle literatury jsou predikční bankrotní modely méně přesné, pokud se použijí v rámci alternativních podmínek. V našem předchozím výzkumu jsme vytvořili vlastní predikční bankrotní model. Při vytváření modelu jsme se snažili aplikovat přístup odlišný od těch předchozích. Pro vytvoření modelu jsme použili tradiční metodu lineární diskriminační analýzy, avšak za použití pouze transformovaných proměnných s přibližně normálním rozdělení. A co víc, použili jsme pouze takové páry proměnných, kterou jsou většinou negativně korelovány. Podle literatury by tyto faktory měly pozitivně ovlivnit přesnosti modelu. Nicméně je velmi omezené pozornost věnována účinku této aplikace na stabilitu přesnosti modelu. Cílem tohoto příspěvku je analyzovat stabilitu přesnosti modelu v aplikaci v různých časových období nebo oborech. Navíc jsme si stanovili za cíl zkoumat a diskutovat o účinnost postupu, který byl použit k vytvoření modelu.

The aim of this paper is to examine discrimination performance of three bankruptcy prediction models in environments and periods different from the ones utilized by deriving the models. We compared selected models’ accuracy in the original setting and present conditions. Secondary aim was to examine a way of possible increasing of the discrimination performance of models by the recalculation of the classification functions. Discrimination performance of the models and financial ratios was tested on companies operating in manufacturing business. Results conclusively demonstrate that the discrimination accuracy of bankruptcy models deteriorates significantly in different environments. The classification function of each model was recalculated using the data from Czech manufacturing companies. For the adjustment of models’ coefficients the same methods, as used originally by theirs authors, were applied, i.e. the probit method, the linear discrimination analysis and the logit method. The results shown, that the re-estimation of model coefficient could lead to its higher classification accuracy in alternative conditions. We can dedicate that recalculating of the classification rules is one of the ways to increase discrimination performance of the bankruptcy prediction models in different environment.

Cultivation of horticultural species under organic management has increased in importance in recent years. However, the sustainability of this new production method needs to be supported by scientific research, especially in the field of virology. We studied the prevalence of three important virus diseases in agroecosystems with regard to its management system: organic versus non-organic, with and without greenhouse. Prevalence was assessed by means of a Bayesian correlated binary model which connects the risk of infection of each virus within the same plot and was defined in terms of a logit generalized linear mixed model (GLMM). Model robustness was checked through a sensitivity analysis based on different hyperprior scenarios. Inferential results were examined in terms of changes in the marginal posterior distributions, both for fixed and for random effects, through the Hellinger distance and a derived measure of sensitivity. Statistical results suggested that organic systems show lower or similar prevalence than non-organic ones in both single and multiple infections as well as the relevance of the prior specification of the random effects in the inferential process.

The mobility of people, freight and information is fundamental to economic and social activities such as commuting, manufacturing, distributing consumer goods and supplying energy. There are two major problems that arise as a result of mobility. The first is economic cost and the second is environmental impact which is of increasing concern in sustainable development due to emission levels, particularly as a result of car use. This study focuses on constructing a network of induced emissions (NOIEs) by using three models and checking the robustness of NOIEs under varying parameters and models. The three models are Stead’s model, the NAEI model, and Oguchi’s model. This study uses the Swedish city of Borlänge as the case study. Calculating CO2 emissions by constructing the NOIEs using Stead’s model appears to give an underestimation when compared to results from a NOIEs which applies Oguchi’s model. Results when applying the NAEI model in constructing a NOIEs also give an underestimation compared to a NOIEs applying Oguchi’s model. Applying the NAEI model is, however, more accurate than applying Stead’s model in constructing a NOIEs. The outcomes of this study show that constructing a NOIEs is robust using Oguchi’s model. This model is preferable since it takes into account more important variables such as driving behavior and the length of the road segments which have a significant impact when estimating CO2 emissions.

According to some authors, bankruptcy models are less accurate if used in an environment different from the one for which they were designed. This is due to the higher heterogeneity of the data. The accuracy of the prediction model may be improved by reducing this heterogeneity. An alternative solution to the problem of the limited transferability (e.g. robustness) of a model may be to identify the environmental factors that affect the models prediction accuracy and incorporate them into it. This paper presents research of the prediction accuracy of a bankruptcy model in four countries and the correlation found between the development of selected macroeconomic indicators in these countries. It proved possible in this way to identify those macroeconomic factors that show similar correlation patterns between given environments as investigated by the accuracy of the bankruptcy model. These factors include, first and foremost, the performance of the economy, inflation and unemployment. The incorporation into the model of such factors is very likely to increase model robustness. Podle některých autorů se predikční přesnost bankrotních modelů při použití v odlišném prostředí snižuje. Důvodem je vyšší heterogenita dat, jejíž snížením lze dosáhnout zvýšení predikční přesnosti modelu. Alternativní možností řešení omezené přenositelnosti modelů se jeví identifikace faktorů vnějšího prostředí, které ovlivňují predikční přesnost modelů a jejich zakomponování do bankrotního modelu. V tomto článku jsou jednak prezentovány výsledky zkoumání predikční přesnosti bankrotního modelu ve čtyřech zemích a dále výsledky zkoumání korelace ve vývoji vybraných makroekonomických ukazatelů v těchto zemích. To umožnilo identifikovat makroekonomické faktory, které mezi danými prostředími vykazují podobné korelační vzory jako přesnost zkoumaného bankrotního modelu. Těmito faktory jsou zejména výkon ekonomiky, inflace a nezaměstnanost. Je pak velmi pravděpodobné, že zakomponování těchto faktorů do modelu povede ke zvýšení robustnosti modelu.

Calibration and validation techniques are crucial in assessing the descriptive and predictive power of car-following models and their suitability for analyzing traffic flow. Using real and generated floating-car and trajectory data, we systematically investigate following aspects: Data requirements and preparation, conceptional approach including local maximum-likelihood and global LSE calibration with several objective functions, influence of the data sampling rate and measuring errors, the effect of data smoothing on the calibration result, and model performance in terms of fitting quality, robustness, parameter orthogonality, completeness and plausible parameter values., Presentation at the 20th ISTTT in Noordwijk, the Netherlands, 2013

Desarrollar una metodología conceptualmente consistente para evaluar y medir el grado de supervivencia de nuevas empresas, basado en el análisis de jóvenes empresas portuguesas, tratando de predecir el riesgo de mortalidad empresarial, la evaluación de la "vida" de su estado de "salud", medidas que prescriben que sirven para prolongar, controlar y mejorar la supervivencia de empresas de nueva creación. Identificar predictores del riesgo de fracaso de una startup. Analizado desde cuatro ejes diferentes: de tres no financiero (de acuerdo con el análisis dimensional del Emprendedor, Corporativo y Medios de Comunicación de Medio Ambiente) y financiero. Los ejes tienen por objeto captar la puesta en marcha de la realidad, en una visión de 360 º grados, de acuerdo a tres niveles (sin riesgo, algo de riesgo y de alto riesgo). De este modo, a través de los resultados de proponer un conjunto de directrices para nuevas empresas, sostenidas en METRISUP – Métrica de Mensuración de Riesgos en Startups, permite un fácil y buena lectura con el fin de minimizar y o anticipar el riesgo de quiebra de negocios/bancarrota en sus organizaciones., Development a conceptually consistent methodology to evaluate and measure the survival rate of startups, based on the analysis of young Portuguese companies. We try to fulfill this objective by looking for ways to predict the risk of business “mortality”, by evaluating companies’ “health condition”, and by prescribing measures to lengthen, control and improve the survival rate of startups. Identify the predictors of failure risk for startups. We analyzed these predictors from four distinct perspectives: three non-financial (according to a dimensional analysis of the Entrepreneur, the Enterprise and its Surroundings) and a financial perspective. The axes seek to capture the reality of a startup, with a 360-degree view, from three levels (no risk, some risk, and high risk). Using the results obtained, was to propose a set of guidelines for startups, supported by the METRISUP model - Risk Assessment Models for Startups, with a friendly-user construction and easy to interpret - in order to minimize the bankruptcy risk in your organization, Desenvolver uma metodologia conceptualmente consistente que permita avaliar e mensurar o grau de sobrevivência de startups, tendo como base a análise de jovens empresas portuguesas, procurando prever o risco de mortalidade empresarial, avaliando em “vida” o seu estado de “saúde”, prescrevendo medidas que permitam prolongar, controlar e melhorar a sobrevida das startups. Identificar fatores preditores do risco de falência de uma startup. Analisados sob quatro eixos distintos: três não financeiros (segundo a análise dimensional do Empreendedor, da Empresa e do Meio Envolvente) e um financeiro. Os eixos pretendem captar a realidade da startup, numa visão de 360º, segundo três níveis (sem risco, algum risco e risco elevado). Assim, através dos resultados obtidos propor um conjunto de orientações para as startups, sustentadas no modelo METRISUP - Métrica de Risco em Startups, de construção fácil e de boa leitura com a finalidade de minimizar o risco de falência nas suas organizações.

Watson and Holmes propose ways of investigating robustness of statistical decisions by examining certain neighbourhoods around a posterior distribution. This may partly amount to ad hoc modelling of extra uncertainty. Instead of creating neighbourhoods around the posterior a posteriori, we argue that it might be more fruitful to model a layer of extra uncertainty first, in the model building process, and then allow the data to determine how big the resulting neighbourhoods ought to be. We develop and briefly illustrate a general strategy along such lines.

International audience

To evaluate the impact of climate change on water resources at the catchment scale, not only future projections of climate are necessary but also robust rainfall-runoff models that must be fairly reliable under changing climate conditions. The aim of this study was thus to assess the robustness of three conceptual rainfall-runoff models (GR4j, HBV and IHACRES) on five basins in northern Tunisia under long-term climate variability, in the light of available future climate scenarios for this region. The robustness of the models was evaluated using a differential split sample test based on a climate classification of the observation period that simultaneously accounted for precipitation and temperature conditions. The study catchments include the main hydrographical basins in northern Tunisia, which produce most of the surface water resources in the country. A 30-year period (1970–2000) was used to capture a wide range of hydro-climatic conditions. The calibration was based on the Kling-Gupta Efficiency (KGE) criterion, while model transferability was evaluated based on the Nash-Sutcliffe efficiency criterion and volume error. The three hydrological models were shown to behave similarly under climate variability. The models simulated the runoff pattern better when transferred to wetter and colder conditions than to drier and warmer ones. It was shown that their robustness became unacceptable when climate conditions involved a decrease of more than 25% in annual precipitation and an increase of more than +1.75 °C in annual mean temperatures. The reduction in model robustness may be partly due to the climate dependence of some parameters. When compared to precipitation and temperature projections in the region, the limits of transferability obtained in this study are generally respected for short and middle term. For long term projections under the most pessimistic emission gas scenarios, the limits of transferability are generally not respected, which may hamper the use of conceptual models for hydrological projections in northern Tunisia.

Statistical inference deals with natural variation between units and with uncertainty due to sampling, measurement error and random assignment to treatments. Almost always it does so within one model, assumed fixed and valid. Here, however, variation of conclusions due to variation in model choice will be discussed. Heuristics to find appropriate models are seldom taught, and model fit tests and model fit diagnostics are far from optimal. The use of more than one model for the same data is illustrated by examples from Item Response Theory.

Consider the multivariate linear model for the random matrixYn×p∼MN(XB,V⊗Σ), whereBis the parameter matrix,Xis a model matrix, not necessarily of full rank, andV⊗Σ is annp×nppositive-definite dispersion matrix. This paper presents sufficient conditions on the positive-definite matrixVsuch that the statistics for testingH0:CB=0vsHa:CB≠0have the same distribution as under the i.i.d. covariance structureI⊗Σ.

We examine a ∞exible class of nonstationary stochastic models for multivariate spatial data proposed by Majumdar et al. (2010). This covariance model is based on convolutions of spatially varying covariance kernels with cen- ters corresponding to the centers of \local stationarity". A Bayesian method for estimation of the parameters in the model based on a Gibbs sampler is applied to simulated data to check for model sensitivity. The efiect of a perturbation of the model in terms of kernel centers is also examined. Finally, the method is applied to a bivariate soil chemistry data from the the Central Arizona Phoenix Long Term Ecological Project (CAP LTER). Prediction bias, prediction standard deviation and predictive coverage are examined for difierent candidate models. In addition, a comparison with the bivariate stationary coregionalization model introduced by Wackernagel (2003) is carried out. A variant of the model proposed in Majumdar et al. (2010), with random kernel centers, is also examined. The latter model is seen to work much better than the stationary coregionalization model, and to per- form comparably with the model with random kernel centers. Simulations indicate that the model is sensitive to under- or over-speciflcation of kernel centers. On the other hand, application to real data seems to indicate that centroids of the regions that are homogeneous can be used as means of the random kernel centers. Cross validation can be used as a way of flnding the best model with an appropriate number of kernels.

Although prospective logistic regression is the standard method of analysis for case-control data, it has been recently noted that in genetic epidemiologic studies one can use the ``retrospective'' likelihood to gain major power by incorporating various population genetics model assumptions such as Hardy-Weinberg-Equilibrium (HWE), gene-gene and gene-environment independence. In this article we review these modern methods and contrast them with the more classical approaches through two types of applications (i) association tests for typed and untyped single nucleotide polymorphisms (SNPs) and (ii) estimation of haplotype effects and haplotype-environment interactions in the presence of haplotype-phase ambiguity. We provide novel insights to existing methods by construction of various score-tests and pseudo-likelihoods. In addition, we describe a novel two-stage method for analysis of untyped SNPs that can use any flexible external algorithm for genotype imputation followed by a powerful association test based on the retrospective likelihood. We illustrate applications of the methods using simulated and real data., Comment: Published in at http://dx.doi.org/10.1214/09-STS297 the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Optimal stopping and mathematical finance are intimately connected since the value of an American option is given as the solution to an optimal stopping problem. Such a problem can be viewed as a game in which we are trying to maximize an expected reward. The solution involves finding the best possible strategy, or equivalently, an optimal stopping time for the game. Moreover, the reward corresponding to this optimal time should be determined. It is also of interest to know how the solution depends on the model parameters. For example, when pricing and hedging an American option, the volatility needs to be estimated and it is of great practical importance to know how the price and hedging portfolio are affected by a possible misspecification. The first paper of this thesis investigates the performance of the delta hedging strategy for a class of American options with non-convex payoffs. It turns out that an option writer who overestimates the volatility will obtain a superhedge for the option when using the misspecified hedging portfolio. In the second paper we consider the valuation of a so-called stock loan when the lender is allowed to issue a margin call. We show that the price of such an instrument is equivalent to that of an American down-and-out barrier option with a rebate. The value of this option is determined explicitly together with the optimal repayment strategy of the stock loan. The third paper considers the problem of how to optimally stop a Brownian bridge. A finite horizon optimal stopping problem like this can rarely be solved explicitly. However, one expects the value function and the optimal stopping boundary to satisfy a time-dependent free boundary problem. By assuming a special form of the boundary, we are able to transform this problem into one which does not depend on time and solving this we obtain candidates for the value function and the boundary. Using stochastic calculus we then verify that these indeed satisfy our original problem. In the fourth paper we consider an investor wanting to take advantage of a mispricing in the market by purchasing a bull spread, which is liquidated in case of a market downturn. We show that this can be formulated as an optimal stopping problem which we then, using similar techniques as in the third paper, solve explicitly. In the fifth and final paper we study convexity preservation of option prices in a model with jumps. This is done by finding a sufficient condition for the no-crossing property to hold in a jump-diffusion setting.

We investigate the robustness of nonlinear filtering for continuous time finite state Markov chains, observed in white noise, with respect to misspecification of the model parameters. It is shown that the distance between the optimal filter and that with incorrect model parameters converges to zero uniformly over the infinite time interval as the misspecified model converges to the true model, provided the signal obeys a mixing condition. The filtering error is controlled through the exponential decay of the derivative of the nonlinear filter with respect to its initial condition. We allow simultaneously for misspecification of the initial condition, of the transition rates of the signal, and of the observation function. The first two cases are treated by relatively elementary means, while the latter case requires the use of Skorokhod integrals and tools of anticipative stochastic calculus., Comment: Published at http://dx.doi.org/10.1214/105051606000000871 in the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Vital to the effectiveness of simulation-based design is having a model of known quality of the system being designed. The purpose of this paper is to validate a simplified dynamic model of an FMTV (Family of Medium Tactical Vehicles) for a range of system parameters using a previously developed technique for determining model robustness and accuracy within a design space. The literature provides an algorithm called AVASIM (Accuracy and Validity Algorithm for Simulation) for assessing model validity systematically and quantitatively. AVASIM assess the validity of a model based on a specific input and set of system parameters. The literature also defines a procedure for evaluating the robustness and accuracy of a model with respect to input and system parameter variations based on the AVASIM algorithm. This procedure produces a Range of Validity for a model that is quantitative, systematic and based on physically meaningful specifications In this paper the Range of Validity techniques are applied to a reduced model of an FMTV in order to verify its accuracy and robustness within a design space. The results from the analysis show that for the design space and simulation conditions chosen a reduced order model of the FMTV vehicle is more than sufficiently accurate. Design decisions can be made with confidence and design optimization studies can be performed faster while using this reduced model. Copyright © 2005 SAE International.

Motivated by an example in nutritional epidemiology, we investigate some design and analysis aspects of linear measurement error models with missing surrogate data. The specific problem investigated consists of an initial large sample in which the response (a food frequency questionnaire, FFQ) is observed, and then a smaller calibration study in which replicates of the error prone predictor are observed (food records or recalls, FR). The difference between our analysis and most of the measurement error model literature is that in our study, the selection into the calibration study can depend upon the value of the response. Rationale for this type of design is given. Two major problems are investigated. In the design of a calibration study, one has the option of larger sample sizes and fewer replicates, or smaller sample sizes and more replicates. Somewhat surprisingly, neither strategy is uniformly preferable in cases of practical interest. The answers depend on the instrument used (recalls or records) and the parameters of interest. The second problem investigated is one of analysis. In the usual linear model with no missing data, method of moments estimates and normal-theory maximum likelihood estimates are approximately equivalent, with the former method in most use because it can be calculated easily and explicitly. Both estimates are valid without any distributional assumptions. In contrast, in the missing data problem under consideration, only the moments estimate is distribution-free, but the maximum likelihood estimate has at least 50% greater precision in practical situations when normality obtains. Implications for the design of nutritional calibration studies are discussed.

Motivated by an example in nutritional epidemiology, we investigate some design and analysis aspects of linear measurement error models with missing surrogate data. The specific problem investigated consists of an initial large sample in which the response (a food frequency questionnaire, FFQ) is observed, and then a smaller calibration study in which replicates of the error prone predictor are observed (food records or recalls, FR). The difference between our analysis and most of the measurement error model literature is that in our study, the selection into the calibration study can depend upon the value of the response. Rationale for this type of design is given. Two major problems are investigated. In the design of a calibration study, one has the option of larger sample sizes and fewer replicates, or smaller sample sizes and more replicates. Somewhat surprisingly, neither strategy is uniformly preferable in cases of practical interest. The answers depend on the instrument used (recalls or records) and the parameters of interest. The second problem investigated is one of analysis. In the usual linear model with no missing data, method of moments estimates and normal-theory maximum likelihood estimates are approximately equivalent, with the former method in most use because it can be calculated easily and explicitly. Both estimates are valid without any distributional assumptions. In contrast, in the missing data problem under consideration, only the moments estimate is distribution-free, but the maximum likelihood estimate has at least 50% greater precision in practical situations when normality obtains. Implications for the design of nutritional calibration studies are discussed.

The use of proper prior densities in regression models with multivariate non-Normal elliptical error distributions is examined when the scale matrix is known up to a precision factor T, treated as a nuisance parameter. Marginally equivalent models preserve the convenient predictive and posterior results on the parameter of interest B obtained in the reference case of the Normal model and its conditionally natural conjugate gamma prior. Prior densities inducing this property are derived for two special cases of non-Normal elliptical densities representing very different patterns of tail behavior. In a linear framework, so-called semi-conjugate prior structures are defined as leading to marginal equivalence to a Normal data density with a fully natural conjugate prior.

Broadening the stochastic assumptions on the error terms of regression models was prompted by the analysis of linear multivariate t models in Zellner (1976). We consider a possible non-linear regression model under any multivariate elliptical data density, and examine Bayesian posterior and productive results. The latter are shown to be robust with respect to the specific choice of a sampling density within this elliptical class. In particular, sufficient conditions for such model robustness are that we single out a precision factor T2 on which we can specify an improper prior density. Apart from the posterior distribution of this nuisance parameter T 2, the entire analysis will then be completely unaffected by departures from Normality. Similar results hold in finite mixtures of such elliptical densities, which can be used to average out specification uncertainty.

Theory for finding designs in estimating a linear functional of a regression function is developed for classes of regression functions which are infinite dimensional. These classes can be viewed as representing possible departures from an "ideal" simple model and thus describe a model robust setting. The estimates are restricted to be linear and the design (and estimate) sought is minimax for mean square error. The structure of the design is obtained in a variety of cases; some asymptotic theory is given when the functionals are integrals. As to be expected, optimal designs depend critically on the particular functional to be estimated. The associated estimate is generally not a least squares estimate but we note some examples where a least squares estimate, in conjunction with a good design, is adequate.