Search

In the design of distributed quantization systems, one inevitably confronts two types of constraints - those imposed by a distributed system's structure and those imposed by how the distributed system is optimized. Structural constraints are inherent properties of any distributed quantization system and are normally summarized by functional relationships defining the inputs and outputs of their component quantizers. The use of suboptimal optimization methods are often necessitated by the computational complexity encountered in distributed problems. This correspondence briefly explores the impact and interplay of these two types of constraints in the context of distributed quantization for detection. We introduce two structures that exploit interquantizer communications and that represent extremes in terms of their structural constraints. We then develop a sequential optimization scheme that maximizes the Kullback-Leibler divergence, takes advantage of statistical dependencies in the distributed system's output variables, and leads to simple parameterizations of the component quantization rules. We present an illustrative example from which we draw insights into how these constraints influence the quantization boundaries and affect performance relative to lower and upper bounds.

The capacity defines the ultimate fidelity limits of information transmission by any system. We derive the capacity of parallel Poisson process channels to judge the relative effectiveness of neural population structures. Because the Poisson process is equivalent to a Bernoulli process having small event probabilities, we infer the capacity of multi-channel Poisson models from their Bernoulli surrogates. For neural populations wherein each neuron has individual innervation, inter-neuron dependencies increase capacity, the opposite behavior of populations that share a single input. We use Shannon's rate-distortion theory to show that for Gaussian stimuli, the mean-squared error of the decoded stimulus decreases exponentially in both the population size and the maximal discharge rate. Detailed analysis shows that population coding is essential for accurate stimulus reconstruction. By modeling multi-neuron recordings as a sum of a neural population, we show that the resulting capacity is much less than the population's, reducing it to a level that can be less than provided with two separated neural responses. This result suggests that attempting neural control without spike sorting greatly reduces the achievable fidelity. In contrast, single-electrode neural stimulation does not incur any capacity deficit in comparison to stimulating individual neurons.

Sensory systems often use groups of redundant neurons to represent stimulus information both during transduction and population coding of features. This redundancy makes the system more robust to corruption in the representation. We approximate neural coding as a projection of the stimulus onto a set of vectors, with the result encoded by spike trains. We use the formalism of frame theory to quantify the inherent noise reduction properties of such population codes. Additionally, computing features from the stimulus signal can also be thought of as projecting the coefficients of a sensory representation onto another set of vectors specific to the feature of interest. The conditions under which a combination of different features form a complete representation for the stimulus signal can be found through a recent extension to frame theory called ''frames of subspaces.'' We extend the frame of subspaces theory to quantify the noise reduction properties of a collection of redundant feature spaces.

During the stationary portion of neuron's spiking response to a stimulus, the stimulus could be coded in the average rate and, more elaborately, in the statistics of the sequence of interspike intervals. We use information processing theory to explicitly define when interval coding occurs and quantify the coding gain beyond rate coding provided by the interval code. We explicitly find the interval distribution commensurate with average rate coding. When we analyzed optomotor neural responses recorded from the crayfish eye, we found little interval coding occurring despite stimulus-induced changes from a unimodal to a bimodal interval distribution.

We consider the problem of digital communication in a Rayleigh flat-fading environment using a multiple-antenna system, when the channel state information is available neither at the transmitter nor at the receiver. It is known that at high signal-to-noise ratio (SNR), or when the coherence interval is much larger than the number of transmit antennas, a constellation of unitary matrices can achieve the capacity of the noncoherent system. However, at low SNR, high spectral efficiencies, or for small values of coherence interval, the unitary constellations lose their optimality and fail to provide an acceptable performance. In this work, inspired by the Stein's lemma, we propose to use the Kullback-Leibler (KL) distance between conditional distributions to design space-time constellations for noncoherent communication. In fast fading, i.e., when the coherence interval is equal to one symbol period and the unitary construction provides only one signal point, the new design criterion results in pulse amplitude modulation (PAM)-type constellations with unequal spacing between constellation points. We also show that in this case, the new design criterion is equivalent to design criteria based on the exact pairwise error probability and the Chernoff information. When the coherence interval is larger than the number of transmit antennas, the resulting constellations overlap with the unitary constellations at high SNR, but at low SNR they have a multilevel structure and show significant performance improvement over unitary constellations of the same size. The performance improvement becomes especially more significant when an appropriately designed outer code or multiple receive antennas are used. This property, together with the facts that the proposed constellations eliminate the need for training sequences and are most suitable for low SNR, makes them a good candidate for uplink communication in wireless systems.

The voltage-source equivalent was first derived by Hermann von Helmholtz (1821-1894) in an 1853 paper. Exactly thirty years later in 1883, Leon Charles Thevenin (1857-1926) published the same result, apparently unaware of Helmholtz's work. The generality of the equivalent source network was not appreciated until forty-three years later. Then, in 1926, Edward Lawry Norton (1898-1983) wrote an internal Bell Laboratory technical report that described in passing the usefulness in some applications of using the current-source form of the equivalent circuit. In that same year, Hans Ferdinand Mayer (1895-1980) published the same result and detailed it fully. As detailed subsequently, these people intertwine in interesting ways.

This paper describes the development of the voltage-source equivalent circuit. A subsequent paper concerns the current-source equivalent and summarizes the story. The formal roots of equivalent circuits are Ohm's Law, Kirchoff's Laws, and the Principle of Superposition.

Motoneuron responses were elicited by global visual motion and stepwise displacements of an illuminated stripe. Stimulus protocols were identical to those used in previous behavioral studies of compensatory eyestalk reflexes. The firing rates and directional selectivity of the motoneuron responses were measured with respect to four stimulus dimensions (spatial frequency, contrast, angular displacement and velocity). The directional selectivity of the motoneuron response was correlated to the previously measured gain of the reflex for each stimulus dimension. The information theoretical analysis is based upon Kullback-Leibler (K-L) distances which measure the dissimilarity of responses to different stimuli. K-L distances for single neurons are strongly influenced by the mean rate difference of the responses to any pair of stimuli. Because of redundancy, the joint K-L distances of pairs of neurons were less than the sum of the K-L distances of the individual neurons. Furthermore, the joint K-L distances were only weakly influenced by correlations among coactivated neurons. For most of the stimulus dimensions, the K-L distances of single motoneurons were not sufficient to account for the stimulus discriminations exhibited by the eyestalk reflex which typically required the summed output of 2 to 5 motoneurons. Thus the behaviorally relevant information is encoded in the motoneuron ensemble. The minimum time required to discriminate the direction of motion (the encoding window) for a single motoneuron is about 380 to 480 ms (including a 175 ms response latency) for stepwise displacements and up to 1.0 s for global motion. During this period a motoneuron fires 2 to 3 impulses.

Signals that represent information may be classified into two forms: numeric and symbolic. Symbolic signals are discrete-time sequences that at, any particular index, have a value that is a member of a finite set of symbols. Set membership defines the only mathematical structure that symbolic sequences satisfy. Consequently, symbolic signals cannot be directly processed with existing signal processing algorithms designed for signals having values that are elements of a field (numeric signals) or a group. Generalizing an approach due to Stoffer (see Biometrika, vol.85, p.201-213, 1998), we extend time-frequency and time-scale analysis techniques to symbolic signals and describe a general linear approach to developing processing algorithms for symbolic signals. We illustrate our techniques by considering spectral and wavelet analyses of DNA sequences.

Compensatory optomotor reflexes were examined in crayfish (Procambarus clarkii) with oscillating sine wave gratings and step displacements of a single stripe. A capacitance transducer was used to measure the rotation of the eyestalk about its longitudinal axis. System studies reveal a spatial frequency response independent of velocity and stimulus amplitude and linear contrast sensitivity similar to that of neurons in the visual pathway. The reflex operates at low temporal frequencies (

Toward localizing high-frequency tonal signals in the horizontal plane, the lateral superior olive (LSO) encodes the interaural level differences, the crucial cue for azimuthal angle extraction by more central nuclei in the binaural pathway. This paper derives optimal processors for localization using inputs from the LSO and anteroventral cochlear nucleus. Both Poisson and non-Poisson point processes are developed to model cellular responses. The LSO projection is shown to be the only input needed in maximum likelihood angle estimation, and it is processed similarly under different LSO response models. However, localization performance depends strongly on the model’s regularity, with increasing regularity improving localization accuracy and precision.

In an attempt to gain insight into the design of linear detectors for additive white noise channels (discrete-time case), we describe several procedures, both optimal and suboptimal. Using the Wiener representation for nonlinear systems, we derive an ad hoc suboptimal design procedure. Exact designs are found when the noise amplitude's probability distribution is stable and when the noise is Laplacian. Considering all the linear detectors thus derived, no general form for the optimal linear detector's unit-sample response becomes apparent. Performance analyses and simulations indicate substantial performance losses occur when linear detectors are used instead of optimal (likelihood ratio) ones.

Weave analysis of artist canvas examines x-ray images taken of the paintings. Algorithms assume an underlying regularity of the canvas weave over short distances and exploits shortspace spectral analysis to determine the fundamental frequency of the horizontal and vertical thread regularity. However, many paintings are too large to be covered by a single x-ray. Feature point analysis exploits brushstrokes and composition to merge several x-ray images into a single one, taking into account and removing both spatial and amplitude distortions. Certain master artists used low-quality canvas that is more irregular than the norm. A theoretical study of quasiperiodic signals shows that while the expected spectrum is a peak that broadens as period irregularity increases, sample function spectra have distinct peaks having an envelope equal to the expected spectrum.

LSO units recover from a spike discharge in a characteristic way, modeled by an intrinsic recovery function that is stimulus invariant up to a scaling factor and a shifting constant. Data analysis shows that the effect of increasing excitatory stimulus level can be described by amplifying the intrinsic recovery function and by shifting it toward shorter intervals. The shifting process secondarily interacts with the absolute deadtime to produce the response characteristics of the three LSO unit types. Decreased excitation is clearly distinguished from inhibition, which affects the scaling, but not the time origin, of the recovery. We conclude that both excitatory and inhibitory stimulus levels are encoded in the timing of LSO unit discharges.

In the context of the narrowband array processing problem, robust methods for accurately estimating the spatial correlation matrix using a priori information about the matrix structure are developed. By minimizing the worse case asymptotic variance, robust, structured, maximum-likelihood-type estimates of the spatial correlation matrix in the presence of noises with probability density functions in the in -contamination and Kolmogorov classes are obtained. These estimates are robust against variations in the noise's amplitude distribution. The Kolmogorov class is demonstrated to be the natural class to use for array processing applications, and a technique is developed to determine exactly the size of this class. Performance of bearing estimation algorithms improves substantially when the robust estimates are used, especially when nonGaussian noise is present. A parametric structured estimate of the spatial correlation matrix that allows direct estimation of the arrival angles is also demonstrated. >

The optimal procedure for detecting the presence of discrete-time signals in additive noise can be derived from the likelihood ratio test. When the noise has statistically independent, identically distributed components, the dependence of the detector's performance on signal characteristics can be related to the Kullback-Leibler (KL) distance between the distributions governing the hypotheses. Performance predictions based on the central limit theorem are shown to be poor approximations to the true performance. Performance of the optimal detector has long been known to increase exponentially with increasing KL distance. Symmetric noise amplitude distributions yield a symmetric dependence on the difference between the signals' amplitudes at each time index. Small-signal (locally optimal) detection performance is shown to depend on signal energy, whereas large-signal performance depends on the signal waveform. When a distance measure can be defined, performance depends on a different measure than that used in the detector with one exception (the Gaussian). >

Two-photon calcium imaging is an emerging experimental technique that enables the study of information processing within neural circuits in vivo. While the spatial resolution of this technique permits the calcium activity of individual cells within the field of view to be monitored, inferring the precise times at which a neuron emits a spike is challenging because spikes are hidden within noisy observations of the neuron's calcium activity. To tackle this problem, we introduce the use of sparse approximation methods for recovering spikes from the time-varying calcium activity of neurons. We derive sufficient conditions for exact recovery of spikes with respect to (i) the decay rate of the spike-evoked calcium event and (ii) the maximum firing rate of the cell under test. We find--both in theory and in practice--that standard sparse recovery methods are not sufficient to recover spikes from noisy calcium signals when the firing rate of the cell is high, suggesting that in order to guarantee exact recovery of spike times, additional constraints must be incorporated into the recovery procedure. Hence, we introduce an iterative framework for structured sparse approximation that is capable of achieving superior performance over standard sparse recovery methods by taking into account knowledge that spikes are non-negative and also separated in time. We demonstrate the utility of our approach on simulated calcium signals in various amounts of additive Gaussian noise and under different degrees of model mismatch.

Correlated non-Gaussian Markov sequences can be considered as filtered white noise (independent, identically distributed sequences of random variables), the filter being a nonlinear system in general. The authors discuss the applicability of linear models and nonlinear methods based on the diagonal series expansion of bivariate densities for analyzing this system. Non-Gaussian sequences exhibit different properties in the forward and backward directions of time. The authors explore the connection to system modeling of this temporal asymmetry and some of its consequences. As an example, they analyze a first-order linear autoregressive model with hyperbolic secant amplifier distribution at its output. >

The problem of determining the maximum number of narrowband signals whose parameters can be estimated with a linear array of M equally spaced sensors is examined. While this number previously has been taken to be M-1 when the signals are mutually uncorrelated, it is shown how to estimate directions and amplitudes for as many as 2M-1 signals. This over twofold increase is accomplished by retaining snapshot-to-snapshot phase information usually lost in algorithms based on spatial correlation matrices. The approach uses length 2M real signal vectors rather than the usual M complex vectors. It is shown that 2M of these real vectors are linearly independent with probability one, and, thus, in the presence of additive white noise, the parameters of 2M-1 signals can be estimated. An algorithm for determining directions and amplitudes is presented. Because of the algorithm's computational complexity, its application is limited to small M and low time-bandwidth products. >

While evidence indicates that neural systems may be employing sparse approximations to represent sensed stimuli, the mechanisms underlying this ability are not understood. We describe a locally competitive algorithm (LCA) that solves a collection of sparse coding principles minimizing a weighted combination of mean-squared error and a coefficient cost function. LCAs are designed to be implemented in a dynamical system composed of many neuron-like elements operating in parallel. These algorithms use thresholding functions to induce local (usually one-way) inhibitory competitions between nodes to produce sparse representations. LCAs produce coefficients with sparsity levels comparable to the most popular centralized sparse coding algorithms while being readily suited for neural implementation. Additionally, LCA coefficients for video sequences demonstrate inertial properties that are both qualitatively and quantitatively more regular (i.e., smoother and more predictable) than the coefficients produced by greedy algorithms.

A model-order determination procedure for time series is described which is based on computing the differential entropy of the time series' multivariate amplitude probability density. The procedure, termed the differential entropy method, applies to linear and nonlinear models for the time series. Simulations were performed to determine the performance characteristics of the differential entropy method on first-order autoregressive models with Gaussian and non-Gaussian inputs and on a first-order nonlinear model. It produces accurate model-order estimates for the linear model once the correlation coefficient becomes large enough. The threshold for accurate estimates is dependent on the amplitude distribution of the model's input. This technique estimates the model order of data having nonlinear dependence structure more accurately than thead hoc application of the AIC and MDL methods. Extension of the method to higher-order models is described, but, because of computation complexity issues, the differential entropy method is best used for estimating small model orders.

A new sphericity test statistic is proposed for the complex case with a distribution that approaches the asymptotic chi-squared distribution much faster than previously proposed statistics, especially for situations in which the sources are very difficult to detect, such as closely spaced or weak sources. Consequently, the probability of a correct estimate is superior to those resulting from other sphericity tests. The capability of this new sphericity test to detect closely space sources is examined and compared to the resolution capabilities of bearing estimation algorithms as well as other well-known source detection algorithms. >

Practical sparse approximation algorithms (particularly greedy algorithms) suffer two significant drawbacks: they are difficult to implement in hardware, and they are inefficient for time-varying stimuli (e.g., video) because they produce erratic temporal coefficient sequences. We present a class of locally competitive algorithms (LCAs) that correspond to a collection of sparse approximation principles minimizing a weighted combination of reconstruction MSE and a coefficient cost function. These systems use thresholding functions to induce local nonlinear competitions in a dynamical system. Simple analog hardware can implement the required nonlinearities and competitions. We show that our LCAs are stable under normal operating conditions and can produce sparsity levels comparable to existing methods. Additionally, these LCAs can produce coefficients for video sequences that are more regular (i.e., smoother and more predictable) than the coefficients produced by greedy algorithms.

In two-hypothesis detection problems with i.i.d. observations, the minimum error probability decays exponentially with the amount of data, with the constant in the exponent equal to the Chernoff distance between the probability distributions characterizing the hypotheses. We extend this result to the general M-hypothesis Bayesian detection problem where zero cost is assigned to correct decisions, and find that the Bayesian cost function's exponential decay constant equals the minimum Chernoff distance among all distinct pairs of hypothesized probability distributions.

In this paper, we outline a manner in which we determine likely models that can describe a given set of neural population response data. We then use this information regarding the likelihood of all possible models to determine the Kullback-Leibler distance between neural population responses to different stimulus conditions.

A signal-processing model for the generation of point processes is derived. In the context of this model, the interval between successive events is treated as a time series. The input-output relationship of a system generating inter-event intervals is shown to be derivable from the intensity of the desired point process. Both stationary and non-stationary processes can be generated with this technique. The input-output characteristic of this system is shown to be nonlinear for most interesting cases.

We describe two multivariate statistical dependence measures which can be orthogonally decomposed to separate the effects of pairwise, triplewise, and higher order interactions between the random variables. These decompositions provide a convenient method of analyzing statistical dependencies between large groups of random variables, within which smaller "sub-groups" may exhibit dependencies separately from the rest of the variables. The first dependence measure is a generalization of Pearson's /spl phi//sup 2/, and we decompose it using an orthonormal series expansion of joint probability density functions. The second measure is based on the Kullback-Leibler distance, and we decompose it using information geometry. Applications of these techniques include analysis of neural population recordings and multimodal sensor fusion. We discuss in detail the simple example of three jointly defined binary random variables.

We develop two new multivariate statistical dependence measures. First, based on the Kullback-Leibler distance, results in a single value that indicates the general level of dependence among the random variables. Second, based on an orthonormal series expansion of joint probability density functions provides more detail about the nature of the dependence. We apply these dependence measures to the analysis of simultaneous recordings made from multiple neurons, in which dependencies are time-varying and potentially information bearing.

Array processing algorithms generally assume that the received signal, composed of both narrowband signals and noise, is Gaussian. This is not true in general. In the context of the narrowband array processing problem, the authors develop robust methods to estimate accurately the spatial correlation matrix. This estimate is robust against amplitude distribution of the noise, be it Gaussian or not, and takes into account sensor placement. Given this estimate, existing array processing algorithms designed for Gaussian circumstances can be used on non-Gaussian problems. The resulting algorithms can be used in the presence of any type of second-order noise process and perform nearly as well as existing algorithms do with Gaussian noise. >

A simple first-order autoregressive model for the generation of non-Gaussian time series is described. It is defined by X/sub n/= rho X/sub n-1/+W/sub n/ and has a hyperbolic secant marginal distribution. This hyperbolic secant model can be used to generate random non-Gaussian sequences which are free of the degeneracy that afflicts the sequences generated using the Laplace model. The generation formula and the bivariate distributions of this model are derived. It is shown that the mean-square (MS) backward prediction error is strictly less than the MS forward prediction error for all first-order autoregressive non-Gaussian models. >

In many situations some information about the structure of the covariance matrix of a random process is known beyond the fact that it is symmetric and positive definite; for instance, the matrix is frequently Toeplitz. Many people have considered the structured covariance matrix estimation problem for Gaussian processes. However, in actual practice, random signals are seldom, if ever, Gaussian. By using a generalization to processes with known non-Gaussian densities, the authors demonstrate how to find the maximum-likelihood estimate of complex Toeplitz covariance matrices and then evaluate the use of this estimate in some passive array beamforming algorithms. There is substantial improvement in the performance of these bearing estimation algorithms when the authors' estimate is used, especially when non-Gaussian noise is present. >

Averaging of the spatial correlation function over the spatial variable-spatial averaging- is performed on the estimated spatial correlation matrix prior to beamforming calculations. Generally speaking, this preprocessing operation reduces the intersignal coherence in the spatial correlation matrix and the variance of the spatial correlation function estimate. Two types of spatial averaging are considered: subaperture averaging and redundancy averaging. Subaperture averaging is shown to significantly reduce coherence only when the source propagation directions are widely spaced while variance reduction is accompanied by a reduction in apparent aperture. In contrast, the reduction of coherence resulting from redundancy averaging is relatively insensitive to source bearing separations. While the aperture is not affected, redundancy averaging may induce bias in the estimated source bearings. >

The authors examine the maximum number of signals whose parameters can be estimated with a linear, equally spaced array of M sensors. The conventional view is that, when none of the signals is fully correlated, this number is one less than the number of sensors; however, the authors show how to make the extension to one less than twice the number of sensors. This increase in the number of signals is accomplished by using length 2M real signal vectors rather than the usual length M complex vectors. It is shown that 2M of these real vectors are linearly independent with probability one, and, thus, angles of arrival can be unambiguously estimated for 2M-1 signals. >

Doppler-time images (DTIs) of a rotating object were formed from the returns of high resolution radars by expressing the Doppler values as a function of time over a span of ranges. Returns from scatterers on the object yielded a characteristic signature of the object's rotation: a section of a sinusoid centered about the negatively sloped zero-crossing. As several scatterers are usually present, these signatures may overlap. The return from each scatterer is noisy and, more importantly, may be missing (falling below the detection threshold) or may be multi-valued (yielding several values exceeding the detection threshold). The Hough transform-the extraction of parametrically expressible features-was used to extract straight line approximations to the return signatures. This algorithm is shown to be insensitive to missing or extraneous values. Quantization in Hough transform computation is shown to determine the sensitivity of the calculation to noisy values. Parameters of the signatures were extracted using a data-dependent clustering algorithm designed to be insensitive to quantization and feature parameterization. >

A type is the histogram estimate of a stationary sequence's amplitude distribution. From training data, types are computed and use to determine which training data best describe subsequently obtained observations. Such type-based detectors are asymptotically optimal in the sense that the maximal exponential error rate is achieved. For digital communication systems using direct-sequence signaling, type-based detectors are shown to be effective. Training data are obtained from preambles, and then used to make individual bit decisions. Simulations show that in this communications scenario, type-based detectors yield nearly optimal performance without any a priori channel information.

Optimal detectors for non-Gaussian channels are nonlinear, and are therefore sensitive to problem parameters. We describe a technique for designing linear detectors for arbitrary additive noise channels that takes into account dependence structure, amplitude distribution, and transmitted signals. This technique is based on Wiener's theory of nonlinear systems, and amounts to cross-correlating the optimal detector's output with its input when it is driven by white Gaussian noise. We have not proven this linear detector's optimality; simulations demonstrate that it has some of the properties required of optimal detectors in several cases. >

Fractal point processes are modeled as doubly stochastic point processes for which the intensity is a fractal stochastic waveform process. With this model, the fractal point process is shown to be self-similar only over long time scales, with short time scales exhibiting little fractal effects. Parameters of the fractal process are measured with the Fano factor: the ratio of the variance to the mean of the number of events occurring in a time interval. The application of the analysis techniques is illustrated by an example taken from single auditory neuron recordings. >

The channel characteristics of practical code division multiple access (CDMA) systems are usually unknown and difficult to model accurately. Type-based receivers, without assuming any a priori model, extract signals successfully from background noise. In this paper, we develop type-based receivers that address two major issues in CDMA signal reception: multiple access interference and multipath fading. We first present the type-based receiver with interference suppression capability, assuming the knowledge of the code and timing of the intended user only. We then show that equal-gain combining (EGC) of type-based statistics is the asymptotically optimal technique for diversity empirical detection. Compared with maximal ratio combining of matched filter outputs, the diversity receiver with EGC of type-based statistics assumes less channel knowledge and yields competitive detection performance in Gaussian noise and better performance in Laplacian noise.

Type-based receivers assume no a priori channel model, and were previously shown to be effective in direct-sequence spread spectrum communications. In this paper, we investigate the macro-diversity combining of multiple type-based receivers in single-mobile-user reception (BPSK). We present a method for the exact calculation of the bit error rate of type-based receivers in this scenario. Maximal ratio combining of type-based receivers gives near optimal performance in Gaussian noise and better performance than combining matched filter outputs in Laplacian noise. The performance gain achieved by diversity reception is significant regardless of noise statistics, especially in the presence of frequency non-selective Rayleigh fading.

We have developed a method for quantifying neural response changes in terms of the Kullback-Leibler distance between the intensity functions for each stimulus condition. We use empirical histogram estimates to characterize the intensity function of the neural response. A critical factor in determining the histogram estimates is selection of bin-width. We analytically derive the Kullback-Leibler distance between two Poisson processes and two dead time modified Poisson processes in terms of the bin-width selected. Our results show that, for constant intensity processes having the same number of expected counts, the distance between the dead time modified processes is larger than between the Poisson processes.

Information processing theory endeavors to quantify how well signals encode information and how well systems, by acting on signals, process information. We use information-theoretic distance measures, the Kullback-Leibler distance in particular, to quantify how well signals represent information. The ratio of distances calculated between two informationally different signals at a system's output and input quantifies the system's information processing properties. Using this approach, we derive the fundamental processing capabilities of simple system architectures that apply universally: the systems and the kinds of signals they process and produce do not affect our general results. Applications in array signal processing and in neural signal analysis illustrate how to apply the theory.

Information processing is performed when a system preserves aspects of the input related to what the input represents while it removes other aspects. To describe a system's information processing capability, input and output need to be compared in a way invariant to the way signals represent information, Kullback-Leibler distance, information-theoretic measure that reflects the data processing theorem, is calculated on the input and output separately and compared to obtain information transfer ratio. We consider the special case where input serves several parallel systems and show that this configuration has the capability to represent the input information without loss. We also derive bounds for asymptotic rates at which the loss decreases as more parallel systems are added and show that the rate depends on the input distribution.

The fundamental questions of sensory neuroscience are how neurons encode information and how neurons extract (process) information. For example, in the auditory system, sound location is determined by a combination of amplitude and time cues. The Lateral Superior Olive (LSO), one of the first nuclei to receive inputs from both ears, is thought to extract high-frequency localization information based on interaural level difference (ILD) and interaural time difference (ITD) cues. However, while the optimal localization processing system produces the same response for the same ILD regardless of source level,1 LSO responses are affected by source level.2 This sensitivity questions the presumption that the LSO is strictly an angle processor, and suggests that the LSO may be coding other stimulus features such as sound level. We have used a new analysis technique to analyze the significance of the LSO sensitivity to sound level compared to that of an optimal angle processor. We also examined the processing of two features: azimuthal angle and source level by the LSO network by comparing the response sensitivity of network inputs with that of its outputs.

Discharge patterns recorded from single auditory‐nerve fibers have demonstrated long‐range dependence, with the count variance‐to‐mean ratio growing as a power of the counting time for times greater than 0.1–1 s. The intent of this study is to provide a large dataset to enable a more detailed investigation of this phenomenon. Based on 108 recordings from a cat, we conclude that the presence of the fractal noise in the discharge rate is independent of characteristic frequency and stimulus level, but does depend on discharge rate. We measured the low‐frequency power of the fractal noise, finding its coefficient of variation to range between 6% and 26% and to decrease as firing rate increases. Such behavior is consistent with multiplicative fractal variations in models of the hair cell membrane permeability to neurotransmitter. Measured standard deviations of spike rate correspond to a sound‐pressure level difference limen of approximately 1 dB.

The behavior of lateral-superior-olive (LSO) auditory neurons over large time scales was investigated. Of particular interest was the determination as to whether LSO neurons exhibit the same type of fractal behavior as that observed in primary VIII-nerve auditory neurons. It has been suggested that this fractal behavior, apparent on long time scales, may play a role in optimally coding natural sounds. We found that a nonfractal model, the nonstationary dead-time-modified Poisson point process (DTMP), describes the LSO firing patterns well for time scales greater than a few tens of milliseconds, a region where the specific details of refractoriness are unimportant. The rate is given by the sum of two decaying exponential functions. The process is completely specified by the initial values and time constants of the two exponentials and by the dead-time relation. Specific measures of the firing patterns investigated were the interspike-interval (ISI) histogram, the Fano-factor time curve (FFC), and the serial count correlation coefficient (SCC) with the number of action potentials in successive counting times serving as the random variable. For all the data sets we examined, the latter portion of the recording was well approximated by a single exponential rate function since the initial exponential portion rapidly decreases to a negligible value. Analytical expressions available for the statistics of a DTMP with a single exponential rate function can therefore be used for this portion of the data. Good agreement was obtained among the analytical results, the computer simulation, and the experimental data on time scales where the details of refractoriness are insignificant. For counting times that are sufficiently large, yet much smaller than the largest time constant in the rate function, the Fano factor is directly proportional to the counting time. The nonstationarity may thus mask fractal fluctuations, for which the Fano factor increases as a fractional power (less than unity) of the counting time.

Fractal intensity point processes--doubly stochastic point processes with a fractal waveform intensity process--are required to describe the discharge patterns recorded from the auditory and visual systems. The Fano factor--the ratio of the variance of the number of events in an interval to the mean of this number--captures the self-similar characteristics of the intensity via two quantities: fractal dimension and fractal time. The fractal dimension is the exponent of the asymptotic power law behavior of the Fano factor with interval duration. The fractal time delineates long-term fractal behavior from short-term characteristics of the data. The average rate and self-similarity parameter of the intensity process, absolute and relative refractory effects, and serial dependence all modify the fractal time. To generate fractal intensity point processes, stochastic fractal processes are derived by applying memoryless, nonlinear transformations to fractional Gaussian noise. The intensity's amplitude distribution in combination with the Fano factor form criteria to choose the transformation that best describes data.

The authors apply the nonparametric kernel predictor to the time-series prediction problem. Because nonparametric prediction makes few assumptions about the underlying time series, it is useful when modeling uncertainties are pervasive, such as when the time series is non-Gaussian. It is shown that the nonparametric kernel predictor is asymptotically optimal for bounded, mixing time series. Numerical experiments were also performed. For the nonlinear autoregressive process, the kernel predictor is shown to outperform greatly the linear predictor; for the Henon time series, the estimated predictor closely resembles the Henon map. >

On the basis of a geometric theory of detection, the authors extend the notion of a signal constellation, a concept deeply rooted in Gaussian problems, to the non-Gaussian case. Significant differences between optimal designs for Gaussian and non-Gaussian situations are shown. In particular, square-wave signals are much more important in heavy-tailed, non-Gaussian noise situations than in Gaussian ones. Furthermore, design guidelines for non-Gaussian problems can vary with the number of signal set members and can depend on SNR. The extent to which suboptimal designs affect performance (using Gaussian-based designs in non-Gaussian situations, for example) can be predicted from calculations of the Kullback information, but only in the sense of determining how the logarithmic error probability rates differ. >

We studied lateral superior olivary (LSO) unit responses to binaural tone-bursts using a general point process approach. We show that inhibition of the ipsilaterally elicited response by contralateral stimulation cannot be modeled simply as a reduction of the ipsilateral input. Statistical analyses reveal that inhibition operates by scaling the intensity of the point process describing the ipsilateral response. In some cases the scaling process has secondary effects: Binaurally elicited discharges produce bimodal interspike interval histograms from units that produce unimodal interval histograms under monaural stimulation. We present a specific point process model that describes the scaling process and successfully replicates the observed responses to monaural and binaural stimulation of the three types of LSO units: slow choppers, fast choppers, and bimodal units. We interpret scaling as a shunting inhibitory process in these LSO neurons. By relating scaling magnitude to interaural level difference, we demonstrate the spatial sensitivity of LSO units.