Search

The Householder algorithm for the QR factorization of a tall thin n x p full-rank matrix X has the added bonus of producing a matrix M with orthonormal columns that are a basis for the orthocomplement of the column space of X. We give a simple formula for M-Transpose x when x is in that orthocomplement. The formula does not require computing M, it only requires the R factor of a QR factorization. This is used to get a remarkably simple computable concrete representation of independent "residuals" in classical linear regression. For Students problem, when p=1, if R(j)=Y(j)-Ybar are the usual (non-independent) residuals, W(j)=R(j+1) - R(1)/(sqrt(n)+1) gives n-1 i.i.d. mean-zero normal variables whose sum of squares is the same as that of the n residuals. Those properties of this formula can (in hindsight) easily be verified directly, yielding a new simple and concrete proof of Student's theorem. It also gives a simple way of generating n-1 exactly mean-zero i.i.d. samples from n samples with unknown mean. Yiping Cheng exhibited concrete linear combinations of the Y(j) with these properties, in the context of a constructive proof of Student's theorem, but that representation is not so simple. Analogous simple results are obtained for regression when there are more predictors, giving a very simple computable concrete formula for n-p i.i.d. independent residuals with the same sum of squares as that of the usual n non-independent residuals. A connection with Cochran's theorem is discussed.

Modern malware typically makes use of a domain generation algorithm (DGA) to avoid command and control domains or IPs being seized or sinkholed. This means that an infected system may attempt to access many domains in an attempt to contact the command and control server. Therefore, the automatic detection of DGA domains is an important task, both for the sake of blocking malicious domains and identifying compromised hosts. However, many DGAs use English wordlists to generate plausibly clean-looking domain names; this makes automatic detection difficult. In this work, we devise a notion of difficulty for DGA families called the smashword score; this measures how much a DGA family looks like English words. We find that this measure accurately reflects how much a DGA family's domains look like they are made from natural English words. We then describe our new modeling approach, which is a combination of a novel recurrent neural network architecture with domain registration side information. Our experiments show the model is capable of effectively identifying domains generated by difficult DGA families. Our experiments also show that our model outperforms existing approaches, and is able to reliably detect difficult DGA families such as matsnu, suppobox, rovnix, and others. The model's performance compared to the state of the art is best for DGA families that resemble English words. We believe that this model could either be used in a standalone DGA domain detector---such as an endpoint security application---or alternately the model could be used as a part of a larger malware detection system., Comment: Accepted to ARES 2019

Deep neural networks (DNNs) are powerful nonlinear architectures that are known to be robust to random perturbations of the input. However, these models are vulnerable to adversarial perturbations--small input changes crafted explicitly to fool the model. In this paper, we ask whether a DNN can distinguish adversarial samples from their normal and noisy counterparts. We investigate model confidence on adversarial samples by looking at Bayesian uncertainty estimates, available in dropout neural networks, and by performing density estimation in the subspace of deep features learned by the model. The result is a method for implicit adversarial detection that is oblivious to the attack algorithm. We evaluate this method on a variety of standard datasets including MNIST and CIFAR-10 and show that it generalizes well across different architectures and attacks. Our findings report that 85-93% ROC-AUC can be achieved on a number of standard classification tasks with a negative class that consists of both normal and noisy samples., Comment: Submitted to ICML 2017

We present a novel hashing strategy for approximate furthest neighbor search that selects projection bases using the data distribution. This strategy leads to an algorithm, which we call DrusillaHash, that is able to outperform existing approximate furthest neighbor strategies. Our strategy is motivated by an empirical study of the behavior of the furthest neighbor search problem, which lends intuition for where our algorithm is most useful. We also present a variant of the algorithm that gives an absolute approximation guarantee; to our knowledge, this is the first such approximate furthest neighbor hashing approach to give such a guarantee. Performance studies indicate that DrusillaHash can achieve comparable levels of approximation to other algorithms while giving up to an order of magnitude speedup. An implementation is available in the mlpack machine learning library (found at http://www.mlpack.org)., Comment: Submitted to SISAP 2016