Search

Statistical inference, such as hypothesis testing and calculating a confidence interval, is an important tool for accessing uncertainty in machine learning and statistical problems. Stochastic gradient methods, such as stochastic gradient descent (SGD), have recently been successfully applied to point estimation in large scale machine learning problems. In this work, we present novel stochastic gradient methods for statistical inference in large scale machine learning problems. Unregularized M -estimation using SGD. Using SGD with a fixed step size, we demonstrate that the average of such SGD sequences can be used for statistical inference, after proper scaling. An intuitive analysis using the Ornstein-Uhlenbeck process suggests that such averages are asymptotically normal. From a practical perspective, our SGD-based inference procedure is a first order method, and is well-suited for large scale problems. To show its merits, we apply it to both synthetic and real datasets, and demonstrate that its accuracy is comparable to classical statistical methods, while requiring potentially far less computation. Approximate Newton-based statistical inference using only stochastic gradients for unregularized M -estimation. We present a novel inference framework for convex empirical risk minimization, using approximate stochastic Newton steps. The proposed algorithm is based on the notion of finite differences and allows the approximation of a Hessian-vector product from first-order information. In theory, our method efficiently computes the statistical error covariance in M -estimation for unregularized convex learning problems, without using exact second order information, or resampling the entire data set. In practice, we demonstrate the effectiveness of our framework on large-scale machine learning problems, that go even beyond convexity: as a highlight, our work can be used to detect certain adversarial attacks on neural networks. High dimensional linear regression statistical inference using only stochastic gra- dients. As an extension of the approximate Newton-based statistical inference algorithm for unregularized problems, we present a similar algorithm, using only stochastic gradients, for statistical inference in high dimensional linear regression, where the number of features is much larger than the number of samples. Stochastic gradient methods for time series analysis. We present a novel stochastic gradient descent algorithm for time series analysis, which correctly captures correlation structures in a time series dataset during optimization. Instead of uniformly sampling indices in vanilla SGD, we uniformly sample contiguous blocks of indices, where the block length depends on the dataset.

Over the recent years, an increasing number of novel crystalline, robust materials that is only single or few atom thick have been created and studied. They are atomic scale derivatives of common solid state structures in which the framework connectivity is terminated with ligands along specific axes. The abrupt termination of the ionic/covalent inorganic framework along a particular direction, gives these materials fundamentally different electronic, optical, thermal, and magnetic properties, which can be manipulated via the design of the organic component. These new materials often exhibit advantageous electronic and optoelectronic behavior, making them viable platforms for applications ranging from light emitting phosphors, to thin film field effect transistors, to photovoltaics. Using the titanium chalcogenides as model systems, we systematically studied the relative influence of the major synthetic parameters including temperature, ligand structure, and ligand-to-metal stoichiometry on the preparation of the one dimensional atomic scale derivatives of TiS2. These phases tend to form at high ligand-to-metal ratios and relatively lower temperatures, while the two-dimensional parent lattices are preferred at higher temperature. Although a small change in ligand structure, such as ethylenediamine to propylenediamine will significantly influence the stability of these phases, it will only subtly change the electronic structure. By developing a systematic understanding of the effects of various factors during the synthesis, this work provides a pathway to rationally create new dimensionally reduced materials. Similarly the selenium analogues of the one dimensional derivatives can also be created. These derivatives exhibit a shift from an indirect to a direct band gap and both the sulfide and the selenide system show a 1.4 eV increment in the band gap energy.The intercalation of metal cations in 2D layered materials allows for the discovery of unique electronic, magnetic and correlated properties. We demonstrate that reversible Li intercalation is also achievable in the hybrid organic/inorganic dimensionally-reduced 1D van der Waals solid TiS2(ethylenediamine). Upon intercalation, electrons are injected into the lattice as Ti4+ is reduced to Ti3+ leading to an order of magnitude decrease in electrical resistivity. This reversible intercalation process opens up new opportunities to fine-tune the physical properties in this emerging family of dimensionally-reduced materials.The vanadium chalcogenide systems are also explored. Reaction temperature, ligand choice and precursor chemistry are studied. The two dimensional organic-intercalated VSe2 inorganic lattices dominate at higher temperature region while the lower temperature region exist various cluster phases, instead of extended lattices as in the titanium chalcogenide systems. For example, we discovered V7S8(en)8Cl6, an electron-deficient corner-sharing dicubane structure, having one unpaired electron per formula. Additionally, in the selenium system, V(en)3Se3 readily forms at low temperatures, and this phase consists of V(en)3 2+ cations having Se32- counteranions with no direct V-Se bonding. These clusters represent low-temperature precursor phases towards the two dimensional lattices in these solution phase reactions. The results from this chapter underlines the rich chemistry of the vanadium chalcogenide system and provides a pathway for solution phase synthesis of intercalated two dimensional VS2 and VSe2 phases as well as other low temperature cluster and precursor phases.Overall, using titanium and vanadium chalcogenides as model systems, a solution phase synthesis methodology has been develop to create the original two dimensional organic intercalated transition metal chalcogenides, their atomic scale one dimensional chain derivatives and low temperature cluster and precursor phases. The electronic structures and the Li intercalation of the new phases are studied to understand the correlation of properties and structures, leading towards the design and discovery of more useful functional hybrid materials.