Search

In response to a growing interest in Total Least Squares (TLS) and Errors-In-Variables (EIV) modeling by researchers and practitioners, well-known experts from several disciplines were invited to prepare an overview paper and present it at the third international workshop on TLS and EIV modeling held in Leuven, Belgium, August 27-29, 2001. These invited papers, representing two-thirds of the book, together with a selection of other presented contributions yield a complete overview of the main scientific achievements since 1996 in TLS and Errors-In-Variables modeling. In this way, the book nicely completes two earlier books on TLS (SIAM 1991 and 1997). Not only computational issues, but also statistical, numerical, algebraic properties are described, as well as many new generalizations and applications. Being aware of the growing interest in these techniques, it is a strong belief that this book will aid and stimulate users to apply the new techniques and models correctly to their own practical problems.

This monograph is a slightly revised version of my PhD thesis [86], com­ pleted in the Department of Computer Science at the University of Edin­ burgh in June 1988, with an additional chapter summarising more recent developments. Some of the material has appeared in the form of papers [50,88]. The underlying theme of the monograph is the study of two classical problems: counting the elements of a finite set of combinatorial structures, and generating them uniformly at random. In their exact form, these prob­ lems appear to be intractable for many important structures, so interest has focused on finding efficient randomised algorithms that solve them ap­ proxim~ly, with a small probability of error. For most natural structures the two problems are intimately connected at this level of approximation, so it is natural to study them together. At the heart of the monograph is a single algorithmic paradigm: sim­ ulate a Markov chain whose states are combinatorial structures and which converges to a known probability distribution over them. This technique has applications not only in combinatorial counting and generation, but also in several other areas such as statistical physics and combinatorial optimi­ sation. The efficiency of the technique in any application depends crucially on the rate of convergence of the Markov chain.

Least squares is probably the best known method for fitting linear models and by far the most widely used. Surprisingly, the discrete L 1 analogue, least absolute deviations (LAD) seems to have been considered first. Possibly the LAD criterion was forced into the background because of the com­ putational difficulties associated with it. Recently there has been a resurgence of interest in LAD. It was spurred on by work that has resulted in efficient al­ gorithms for obtaining LAD fits. Another stimulus came from robust statistics. LAD estimates resist undue effects from a feyv, large errors. Therefore. in addition to being robust, they also make good starting points for other iterative, robust procedures. The LAD criterion has great utility. LAD fits are optimal for linear regressions where the errors are double exponential. However they also have excellent properties well outside this narrow context. In addition they are useful in other linear situations such as time series and multivariate data analysis. Finally, LAD fitting embodies a set of ideas that is important in linear optimization theory and numerical analysis. viii PREFACE In this monograph we will present a unified treatment of the role of LAD techniques in several domains. Some of the material has appeared in recent journal papers and some of it is new. This presentation is organized in the following way. There are three parts, one for Theory, one for Applicatior.s and one for Algorithms.

Als mehrbändiges Nachschlagewerk ist das Springer-Handbuch der Mathematik in erster Linie für wissenschaftliche Bibliotheken, akademische Institutionen und Firmen sowie interessierte Individualkunden in Forschung und Lehre gedacht. Es ergänzt das einbändige themenumfassende Springer-Taschenbuch der Mathematik (ehemaliger Titel Teubner-Taschenbuch der Mathematik), das sich in seiner begrenzten Stoffauswahl besonders an Studierende richtet. Teil III des Springer-Handbuchs enthält neben den Kapiteln 5-9 des Springer-Taschenbuchs zusätzliches Material zu stochastischen Prozessen.​

​​​​​​ Stationarity and Convergence in Reduce-or-Retreat Minimization presents and analyzes a unifying framework for a wide variety of numerical methods in optimization. The author's “reduce-or-retreat” framework is a conceptual method-outline that covers any method whose iterations choose between reducing the objective in some way at a trial point, or retreating to a closer set of trial points. The alignment of various derivative-based methods within the same framework encourages the construction of new methods, and inspires new theoretical developments as companions to results from across traditional divides. The text illustrates the former by developing two generalizations of classic derivative-based methods which accommodate non-smooth objectives, and the latter by analyzing these two methods in detail along with a pattern-search method and the famous Nelder-Mead method.In addition to providing a bridge for theory through the “reduce-or-retreat” framework, this monograph extends and broadens the traditional convergence analyses in several ways. Levy develops a generalized notion of approaching stationarity which applies to non-smooth objectives, and explores the roles of the descent and non-degeneracy conditions in establishing this property. The traditional analysis is broadened by considering “situational” convergence of different elements computed at each iteration of a reduce-or-retreat method. The “reduce-or-retreat” framework described in this text covers specialized minimization methods, some general methods for minimization and a direct search method, while providing convergence analysis which complements and expands existing results.​ ​

“The book is outstanding and admirable in many respects.... is necessary reading for all kinds of readers from undergraduate students to top authorities in the field.” Journal of Symbolic Logic Written by two experts in the field, this is the only comprehensive and unified treatment of the central ideas and applications of Kolmogorov complexity. The book presents a thorough treatment of the subject with a wide range of illustrative applications. Such applications include the randomness of finite objects or infinite sequences, Martin-Loef tests for randomness, information theory, computational learning theory, the complexity of algorithms, and the thermodynamics of computing. It will be ideal for advanced undergraduate students, graduate students, and researchers in computer science, mathematics, cognitive sciences, philosophy, artificial intelligence, statistics, and physics. The book is self-contained in that it contains the basic requirements from mathematics and computerscience. Included are also numerous problem sets, comments, source references, and hints to solutions of problems. New topics in this edition include Omega numbers, Kolmogorov–Loveland randomness, universal learning, communication complexity, Kolmogorov's random graphs, time-limited universal distribution, Shannon information and others.