Search

Mobile devices are widely used around the world, frequently by people speaking local languages or dialects that are not well documented. For these languages, it might not be beneficial for commercial companies to develop Automatic Speech Recognition (ASR) systems, so users of these languages cannot utilize voice activation features (often using Keyword Spotting, KWS) of their devices. Standard KWS methods aim to statistically model the generation process of the speech signal, requiring hours of recorded and transcribed speech for training, and therefore are not adequate for limited-data scenarios. In this paper we propose a new KWS method, suitable for limited-data scenarios, which can be easily applied by developers. The proposed method uses a new histogram representation for words, obtained with respect to a pre-trained Gaussian Mixture Model (GMM). Sentences are represented by fixed-length global feature vectors, extracted from the response curves obtained by a word classifier. Word and sentence classifiers are trained using a discriminative approach, which is typically robust to training-set size. The dataset for training the GMM is easy to obtain, since no annotation is required. We compared the proposed system to a Hidden Markov Model (HMM) based system, trained using the same low data-resources conditions as ours, and to a state-of-the-art ASR system, trained using either the limited data scenario, or using many hours of recorded speech. In the limited data situation, our system performs better then both benchmarks in all experiments except for clean speech of children (CSLU dataset), where it performs as good as the HMM. Since the ASR benchmark performs poorly without enough training data, we also trained it without limiting the available data. In this case the ASR benchmark performs better when tested on speech of adults (TED-LIUM dataset of TED lectures) for all noise conditions, and our system performs better when tested on speech of children with low to moderate SNR values. The results demonstrate the advantages of the proposed system, and the conditions under which it performs better.

Traditional beamformer design methods suffer from a frequency varying beamwidth, making them less effective for wideband applications. Existing approaches that try to solve this problem demonstrate degraded beamformer performance in terms of sidelobe level attenuation and sensitivity, making them less robust to interference and array attribute perturbations. Another common issue for existing approaches is high computational complexity, resulting in the ability to implement these approaches only for small microphone arrays, directly affecting the beamformer spatial resolution. In this paper, we introduce a new approach for designing a constant beamwidth beamformer. The proposed approach utilizes custom-tailored finite impulse response (FIR) filters for each microphone channel, manipulating the beampattern to achieve a desired beamwidth. The manipulated beampattern has approximately a constant beamwidth over a wide frequency band. Additionally, a post-summing output normalization filter is used to ensure a frequency invariant gain of the beampattern. By exploiting the physical microphone array configuration and attributes, we shape accordingly the frequency response of the FIR filters and control the beamformer beamwidth. The proposed approach demonstrates improved array response results in various scenarios, compared to available methods in the field, especially in terms of sensitivity to parameters mismatch, noise robustness and sidelobe attenuation. HighlightsA new method for the design of a constant beamwidth beamformer is proposed.The method designs FIR filters to achieve a constant beamwidth beampattern.The FIR filter frequency response is shaped to control the beamformer beamwidth.Smoothing magnitude coefficients were introduced in the FIR filter design.Improved performance is achieved in various scenarios, compared to other methods.

As 3D point cloud acquisition sensors become increasingly prevalent in urban environments (e.g., LiDAR sensors for autonomous vehicles), the need arises to find efficient ways to align large amounts of such 3D data, often in real-time. In this work, we propose a novel method for 3D point cloud registration refinement in an urban environment (e.g., between Terrestrial LiDAR Scans - TLS - and Airborne LiDAR Scans - ALS), assuming an initial coarse registration is available. The proposed method is based on estimation of the direction of gravity, wall detection, projection of the point clouds on a perpendicular horizontal plane, and conversion into 2D edge-maps. Then, two methods are considered for alignment between the 2D edge-maps: a 2D variant of the well-known ICP (Iterative Closest Point) algorithm, and Edge-Map Phase-Correlation (EMPC). We demonstrate the usefulness of the proposed methods for registration in this challenging task, where the 2D variant of ICP achieves a meaningful advantage over 3D ICP in terms of runtime, while maintaining comparable registration accuracy.

Local-to global point cloud registration is a challenging task due to the substantial differences between these two types of data, and the different techniques used to acquire them. Global clouds cover large-scale environments and are usually acquired aerially, e.g., 3D modeling of a city using Airborne Laser Scanning (ALS). In contrast, local clouds are often acquired from ground level and at a much smaller range, for example, using Terrestrial Laser Scanning (TLS). The differences are often manifested in point density distribution, occlusions nature, and measurement noise. As a result of these differences, existing point cloud registration approaches, such as keypoint-based registration, tend to fail. We improve upon a different approach, recently proposed, based on converting the global cloud into a viewpoint-based cloud dictionary. We propose a local-toglobal registration method where we replace the dictionary clouds with viewpoint descriptors, consisting of panoramic range-images. We then use an efficient dictionary search in the Discrete Fourier Transform (DFT) domain, using phase correlation, to rapidly find plausible transformations from the local to the global reference frame. We demonstrate our method’s significant advantages over the previous cloud dictionary approach, in terms of computational efficiency and memory requirements. In addition, We show its superior registration performance in comparison to a state-ofthe- art, keypoint-based method (FPFH). For the evaluation, we use a challenging dataset of TLS local clouds and an ALS large-scale global cloud, in an urban environment.

The use of 3D point clouds is currently of much interest. One of the cornerstones of 3D point cloud research and applications is point cloud registration. Given two point clouds, the goal of registration is aligning them in a common coordinate system. In particular, we seek in this work to align a sparse and noisy local point cloud, created from a single stereo pair of images, to a dense and large-scale global point cloud, representing an urban outdoors environment. The common approach of keypoint-based registration, tends to fail due to the sparsity and low quality of the stereo local cloud. We propose here a new approach. It consists of the creation of a dictionary of much smaller clouds using a grid of synthetic viewpoints over the dense global cloud. We then perform registration via an efficient dictionary search. Our approach shows promising results on data acquired in an urban environment.

A concatenative text-to-speech (CTTS) synthesizer requires a large acoustic database for high-quality speech synthesis. This database consists of many acoustic leaves, each containing a number of short, compressed, speech segments. In this paper, we propose two algorithms for recompression of the acoustic database, by recompressing the data in each acoustic leaf, without compromising the perceptual quality of the obtained synthesized speech. This is achieved by exploiting the redundancy between speech frames and speech segments in the acoustic leaf. The first approach is based on a vector polynomial temporal decomposition. The second is based on 3-D shape-adaptive discrete cosine transform (DCT), followed by optimized quantization. In addition we propose a segment ordering algorithm in an attempt to improve overall performance. The developed algorithms are generic and may be applied to a variety of compression challenges. When applied to compressed spectral amplitude parameters of a specific IBM small footprint CTTS database, we obtain a recompression factor of 2 without any perceived degradation in the quality of the synthesized speech.

Concatenative synthesis and statistical synthesis are the two main approaches to text-to-speech (TTS) synthesis. Concatenative TTS (CTTS) stores natural speech features segments, selected from a recorded speech database. Consequently, CTTS systems enable speech synthesis with natural quality. However, as the footprint of the stored data is reduced, desired segments are not always available in the stored data, and audible discontinuities may result. On the other hand, statistical TTS (STTS) systems, in spite of having a smaller footprint than CTTS, synthesize speech that is free of such discontinuities. Yet, in general, STTS produces lower quality speech than CTTS, in terms of naturalness, as it is often sounding muffled. The muffling effect is due to over-smoothing of model-generated speech features. In order to gain from the advantages of each of the two approaches, we propose in this work to combine CTTS and STTS into a hybrid TTS (HTTS) system. Each utterance representation in HTTS is constructed from natural segments and model generated segments in an interweaved fashion via a hybrid dynamic path algorithm. Reported listening tests demonstrate the validity of the proposed approach.

This paper presents a tree-based framework for producing self-dual morphological operators, based on a tree-representation complete inf-semilattice (CISL). The idea is to use a self-dual tree transform to map a given image into the above CISL, perform one or more morphological operations there, and map the result back to the image domain using the inverse tree transform. We also present a particular case of this general framework, involving a new tree transform, the Extrema-Watershed Tree (EWT). The operators obtained by using the EWT in the above framework behave like classical morphological operators, but in addition are self-dual. Some application examples are provided: pre-processing for OCR and dust and scratch removal algorithms, and image denoising. We also explore first steps towards obtaining tree transforms that induce a CISL on the image domain as well.

In statistical HMM-based text-to-speech systems (STTS), speech feature dynamics is modeled by first- and second-order feature frame differences, which, typically, do not satisfactorily represent frame to frame feature dynamics present in natural speech. The reduced dynamics results in over-smoothing of speech features, often sounding as muffled synthesized speech. In this correspondence, we propose a method to enhance a baseline STTS system by introducing a segment-wise model representation with a norm constraint. The segment-wise representation provides additional degrees of freedom in speech feature determination. We exploit these degrees of freedom for increasing the speech feature vector norm to match a norm constraint. As a result, statistically generated speech features are less over-smoothed, resulting in more natural sounding speech, as judged by listening tests.

In this paper, we address the problem of redundancy reduction of high-dimensional noisy signals that may contain anomaly (rare) vectors, which we wish to preserve. Since anomaly data vectors contribute weakly to the l2-norm of the signal as compared to the noise, l2 -based criteria are unsatisfactory for obtaining a good representation of these vectors. As a remedy, a new approach, named Min-Max-SVD (MX-SVD) was recently proposed for signal-subspace estimation by attempting to minimize the maximum of data-residual l2-norms, denoted as l2,l and designed to represent well both abundant and anomaly measurements. However, the MX-SVD algorithm is greedy and only approximately minimizes the proposed l2,l-norm of the residuals. In this paper we develop an optimal algorithm for the minization of the l2,l-norm of data misrepresentation residuals, which we call Maximum Orthogonal complements Optimal Subspace Estimation (MOOSE). The optimization is performed via a natural conjugate gradient learning approach carried out on the set of n dimensional subspaces in IRm, m ≫ n, which is a Grassmann manifold. The results of applying MOOSE, MX-SVD, and l2- based approaches are demonstrated both on simulated and real hyperspectral data.

This paper presents a model-based transrating (bit-rate reduction) system for H.264 coded video via requantization. In works related to previous standards, optimal requantization step sizes were obtained via Lagrangian optimization that minimizes the distortion subject to a rate constraint. Due to H.264 advanced coding features, the choices of quantization step size and coding modes are dependent and the rate control becomes computationally expensive. Therefore, optimal requantization algorithms developed for previous standards cannot be applied as they are. Hence, previous works on transrating in H.264 focused on changing the input coding decisions rather than on rate control, while requantization was addressed by a simple one-pass algorithm. Here we propose new model-based optimal requantization algorithms for transrating of H.264 coded video. The optimal requantization goal is to achieve the target bit rate with minimal effect on video quality. Incorporation of the proposed models serves two goals. For intra-coded frames, a novel closed-loop statistical estimator that overcomes spatial neighbors dependencies is developed. For inter-coded frames, the proposed macroblock-level models reduce the computational burden of the optimization. Overall, as compared to re-encoding (cascaded decoder-encoder), the proposed system reduces the computational complexity by a factor of about four, at an average PSNR loss of only 0.4 dB for transrating CIF/SIF sequences from 2 Mb/s to 1 Mb/s. In comparison with a simple one-pass requantization, the proposed algorithm achieves better performance (an average PSNR gain of 0.45 dB), at the cost of just twice the complexity.

Audio streaming applications have become very popular in recent years, owing to their low cost and convenience. However, during network congestions, data packets are often delayed or discarded, creating an annoying gap in the streamed media. This letter presents a new approach to audio packet loss concealment designed for MPEG-audio streaming applications. In a previous work, we introduced a receiver-based concealment algorithm based on applying the gapped-data amplitude and phase estimation (GAPES) interpolation algorithm in the discrete short-time Fourier transform (DSTFT) complex domain and obtained better results compared to past methods. The current approach applies the same algorithm on a different complex domain, formed from combining the modified discrete cosine transform (MDCT) domain as its real part and the modified discrete sine transform (MDST) domain as the imaginary part. The new approach significantly reduces the complexity demands while maintaining similar high-quality results.

In this paper, we address the problem of redundancy-reduction of high-dimensional noisy signals that may contain anomaly (rare) vectors, which we wish to preserve. For example, when applying redundancy reduction techniques to hyperspectral images, it is essential to preserve anomaly pixels for target detection purposes. Since rare-vectors contribute weakly to the -norm of the signal as compared to the noise, -based criteria are unsatisfactory for obtaining a good representation of these vectors. The proposed approach combines and norms for both signal-subspace and rank determination and considers two aspects: One aspect deals with signal-subspace estimation aiming to minimize the maximum of data-residual -norms, denoted as , for a given rank conjecture. The other determines whether the rank conjecture is valid for the obtained signal-subspace by applying Extreme Value Theory results to model the distribution of the noise -norm. These two operations are performed alternately using a suboptimal greedy algorithm, which makes the proposed approach practically plausible. The algorithm was applied on both synthetically simulated data and on a real hyperspectral image producing better results than common -based methods.

Vector quantization (VQ) is an effective and widely known method for low-bit-rate communication of speech and image signals. A common assumption in the design of VQ-based communication systems is that the compressed digital information is transmitted through a perfect channel. Under this assumption, quantization distortion is the only factor in output signal fidelity. Moreover, the assignment of channel symbols to the VQ reconstruction vectors is of no importance. However, under physical channels, errors may be present, causing degradation in overall system performance. In such a case, the effect of channel errors on the coding system performance depends on the index assignment of the reconstruction vectors. The index assignment problem is a special case of the Quadratic Assignment Problem (QAP) and is known to be NP-complete. For a VQ with N reconstruction vectors there are N! possible assignments, meaning that an exhaustive search over all possible assignments is practically impossible. To help the VQ designer, we present in this correspondence lower and upper bounds on the performance of VQ systems under channel errors, over all possible assignments. The bounds coincide with a general bound for the QAP. Nevertheless, the proposed derivation allows us to compare the bounds with published results on VQ index assignment. A related expression for the average performance is also given and discussed. Special cases and numerical examples are given in which the bounds and average performance are compared with index assignments obtained by known algorithms.

This work deals with fitting 2D and 3D implicit polynomials (IPs) to 2D curves and 3D surfaces, respectively. The zero-set of the polynomial is determined by the IP coefficients and describes the data. The polynomial fitting algorithms proposed in this paper aim at reducing the sensitivity of the polynomial to coefficient errors. Errors in coefficient values may be the result of numerical calculations, when solving the fitting problem or due to coefficient quantization. It is demonstrated that the effect of reducing this sensitivity also improves the fitting tightness and stability of the proposed two algorithms in fitting noisy data, as compared to existing algorithms like the well-known 3L and gradient-one algorithms. The development of the proposed algorithms is based on an analysis of the sensitivity of the zero-set to small coefficient changes and on minimizing a bound on the maximal error for one algorithm and minimizing the error variance for the second. Simulation results show that the proposed algorithms provide a significant reduction in fitting errors, particularly when fitting noisy data of complex shapes with high order polynomials, as compared to the performance obtained by the above mentioned existing algorithms.

Common voice conversion methods are based on Gaussian Mixture Modeling (GMM), which requires exhaustive training (typically lasting hours), often leading to ill-conditioning if the dataset used is too small. We propose a new conversion method that is trained in seconds, using either small or large scale datasets. The proposed Grid-Based (GB) method is based on sequential Bayesian tracking, by which the conversion process is expressed as a sequential estimation problem of tracking the target spectrum based on the observed source spectrum. The converted MFCC vectors are sequentially evaluated using a weighted sum of the target training set used as grid-points. To improve the perceived quality of the synthesized signals, we use a post-processing block for enhancing the global variance. Objective and subjective evaluations show that the enhanced-GB method is comparable to classic GMM-based methods, in terms of quality, and comparable to their enhanced versions, in terms of individuality.

In this paper we refer to the correlation analysis between the Non-Local Means (NLM) dissimilarity elements within a given search region. This analysis is required for a more accurate determination of a model-based adaptive search region that we introduced earlier. We explore three levels of correlation according to the degree of patches overlap and explain how this analysis can be used in our model-based NLM approach.

Many voice conversion systems require parallel training sets of the source and target speakers. Non-parallel training is more complicated as it involves evaluation of source-target correspondence along with the conversion function itself. INCA is a recently proposed method for non-parallel training, based on iterative estimation of alignmentand conversion function. The alignment is evaluated using a simple nearestneighbor search, which often leads to phonetic miss-matched source-target pairs. We propose here a generalized approach, denoted as Temporal-Context INCA (TC-INCA), based on matching temporal context vectors. We formulate the training stage as a minimization problem of a joint cost, considering both context-based alignment and conversion function. We show that TC-INCA reduces the joint cost and prove its convergence. Experimental results indicate that TC-INCA significantly improves the alignment accuracy, compared to INCA. Moreover, subjective evaluations show that TC-INCA leads to improved quality of the synthesized output signals, when small training sets are used.

A translation-invariant denoising method based on the minimum description length (MDL) criterion and treestructured best-basis algorithms is presented. A collection of signal models is generated using an extended library of orthonormal wavelet-packet bases, and an additive cost function, approximately representing the MDL principle, is derived. We show that the minimum description length of the noisy observed data is achieved by utilizing the shift-invarient wavelet packet decomposition (SIWPD) and thresholding the resulting coe$cients. This approach is extendable to local trigonometric decompositions, and corresponding procedures to optimize either the library of bases or the "lter banks used at each node of the expansion-tree are described. The signal estimator is e$ciently combined with a modi"ed Wigner distribution, yielding robust time}frequency representations, characterized by high resolution and suppressed interference-terms. The proposed method is compared to alternative existing methods, and its superiority is demonstrated by synthetic and real data examples. ( 1999 Elsevier Science B.V. All rights reserved. Zusammenfassung Wir praK sentieren eine translationsinvariante Denoising-Methode, die auf dem Minimum Description Length (MDL) Kriterium und tree-structured best-basis Algorithmen basiert. Wir erzeugen eine Sammlung von Signalmodellen mit Hilfe einer erweiterten Bibliothek von orthonormalen Wavelet-packet Basen. Weiters wird eine additive Kostenfunktion, die naK herungsweise das MDL Prinzip repraK sentiert, abgeleitet. Wir zeigen, da{ die minimale BeschreibungslaK nge der beobachteten verrauschten Daten durch ein Thresholding der shift-invariant Wavelet-packet (SIWPD) Koe$zienten erreicht wird. Dieses Verfahren kann auf lokale trigonometrische Signalentwicklungen erweitert werden. Die entsprechende Optimierung der ZugehoK rigen Bibliothek von Basen bzw. der FilterbaK nke, die an jedem der Knoten des Entwicklungsbaumes verwendet werden, wird in der vorliegenden Arbeit beschrieben. Wir kombinieren unseren SignalschaK tzer mit einer modi"zierten Wignerverteilung. Die daraus resultierenden robusten Zeit}Frequenz Darstellungen haben hohe Au#oK sung und reduzierte Interferenzterme. Das vorgeschlagene Verfahren wird mit alternativen existierenden Verfahren verglichen. Die UG berlegenheit der neuen Methode wird an Hand von synthetischen und natuK rlichen Daten demonstriert. ( 1999 Elsevier Science B.V. All rights reserved.

The Wigner distribution (WD) possesses a number of desirable mathematical properties relevant to time—frequency analysis. However, the presence of interference terms renders the WD of multicomponent signals extremely diƒcult to interpret. In this work, we propose adaptive suppression of interference terms using the shift-invariant wavelet packet decomposition. A prescribed signal is expanded on its best basis and transformed into the Wigner domain. Subsequently, the interference terms are eliminated by adaptively thresholding the cross-WD of interactive basis functions, according to their amplitudes and distance in an idealized time—frequency plane. We define a distance measure that weighs the Euclidean distance with the local distribution of the signal. The amplitude and distance thresholds control the cross-term interference, the useful properties of the distribution, and the computational complexity. The properties of the resultant modified …igner distribution (MWD) are investigated, and its performance in eliminating interference terms, while still retaining high-energy resolution, is compared with that of other existing approaches. It is shown that the proposed MWD is directly applicable to resolving multicomponent signals. Each component is determined as a partial sum of basis functions over a certain equivalence class in the time—frequency plane. ( 1999 Elsevier Science B.V. All rights reserved. Zusammenfassung Die Wigner-Verteilung (WD) besitzt eine Reihe wu‹ nschenswerter mathematischer Eigenschaften, die fu‹ r eine Zeit— Frequenzanalyse von Bedeutung sind. Allerdings erschwert das Auftreten von Kreuztermen die Interpretation von WD mehrkomponentiger Signale extrem. In dieser Arbeit stellen wir eine adaptive Unterdru‹ ckung von Kreuztermen unter Verwendung der »erschiebungsinvarianten …avelet-Paket-Zerlegung vor. Ein vorgeschriebenes Signal wird auf seine beste Basis erweitert und in den Wigner-Bereich transformiert. Anschlie{end werden die Kreuzterme durch einen adaptiven Schwellwertvergleich mit der Kreuz-WD wechselwirkender Basisfunktionen eliminiert, entsprechend ihrer Amplituden und Abstand in einer idealisierten Zeit—Frequenzebene. Wir definieren ein Abstandsma{, das den euklidischen Abstand mit der lokalen Verteilung des Signals gewichtet. Der Amplituden - und Abstandsschwellwert kontrolliert die Kreuztermsto‹ rung, die nu‹ tzlichen Eigenschaften der Verteilung und den Rechenaufwand. Die Eigenschaften der resultierenden modifizierten …igner-»erteilung (MWD) werden untersucht und ihre Leistungsfa‹ higkeit zur Eliminierung der Kreuzterme bei noch immer hoher Energieauflo‹ sung wird mit der anderer Ansa‹ tze verglichen. Es wird gezeigt

In this paper, an extended library of smooth local trigonometric bases is defined, and an appropriate fast “best-basis” search algorithm is introduced. When compared with the standard local cosine decomposition (LCD), the proposed algorithm is advantageous in three respects. First, it leads to a best-basis expansion that is shift-invariant. Second, the resulting representation is characterized by a lower information cost. Third, the polarity of the folding operator is adapted to the parity properties of the segmented signal at the end-points. The shift invariance stems from an adaptive relative shift of expansions in distinct resolution levels. We show that at any resolution level l it suffices to examine and select one of two relative shift options — a zero shift or a 2 −l−1 shift. A variable folding operator, whose polarity is locally adapted to the parity properties of the signal, further enhances the representation. The computational complexity is manageable and comparable to that of the LCD.

The Non-Local Means (NLM) denoising algorithm uses a weighted average of pixels, within a defined search region in an image, to estimate the noise-free pixel value. The search region is usually a rectangular neighborhood, centered at the pixel of interest, which may include pixels whose original gray value do not match the value of the original central pixel. Consequently, their participation in the averaging process degrades denoising performance. To eliminate their effect, researchers suggest creating an adaptive search region which excludes those dissimilar pixels. In this paper, we present a novel model-based method which defines a set of similar pixels, from the initial search region, using the statistical distribution of the dissimilarity measure. Moreover, to enhance the denoising, our method also adaptively assigns one of two dissimilarity kernels to each pixel, based on its local features. Experimental results show that the proposed algorithm has better performance than the original one in terms of PSNR, SSIM, and visual quality and is found to be more efficient than other examined approaches.

A technique for global -motion estimation and compensation in image sequences of 3-D scenes is described in this paper. Each frame is segmented into regions whose motion can be described by a single set of parameters and a set of motion parameters is estimated for each segment. This is done using an iterative block-based image segmentation combined with the estimation of the parameters describing the global motion of each segment. The segmentation is done using a Gibbs-Markov model-based iterative technique for finding a local optimum solution to a maximum a posteriori probability (MAP) segmentation problem. The initial condition for this process is obtained by applying a Hough transform to the motion vectors of each block in the frame obtained by block matching. In each iteration, given a segmentation, the motion parameters are estimated using the least-squares (LS) technique. To obtain the final segmentation and the more appropriate higher-order motion model for each segment, a final stage of splitting/merging of segments is needed. This step is performed on the basis of maximum-likelihood decisions combined with the determination of the higher-order model parameters by LS. The incorporation of the proposed global-motion estimation technique in an image-sequence coder was found to bring about a substantial reduction in bit-rate without degrading the perceived quality or the PSNR.

Distributed Video Coding (DVC) is an emerging coding scheme that employs principles of source coding with side information (SI) at the decoder. In this paper, we present a new DVC encoder, named LORD (LOw-complexity, Rate-controlled, Distributed video coding system). No feedback channel is used in our encoder, and an adaptive noise model that varies both spatially and temporally is employed. The SI is created using motion extrapolation, resulting in a low delay encoding process. We extend LORD for encoding videos acquired by Bayer sensors in endoscopy, in which only partial information of the colors in each pixel is known. This special video format has not been addressed yet in the DVC framework. We show that, using LORD, a significant improvement in performance is achieved over a standard intra-coding method with a similar complexity, on a set of examined videos.

In Hybrid DPCM image sequence coders, every block with high variance in the corresponding difference block is replenished even if it belongs to a stationary background region, as could be the case for coarse textured blocks. We present an improved Hybrid DPCM image sequence coder, which includes a block classification unit. This unit detects stationary textured background blocks and allows copying such blocks instead of replenishing them, thus reducing the bit-rate of the coder. The unit consists of three parts. The first part is a statistical-based change detector. Segmentation of each image into textured and smooth regions, a fast converging deterministic relaxation procedure and a multi-resolution approach are used to obtain the final moving/stationary segmentation. In the second part, a Gaussian AR model-based texture matching test is proposed. The third part detects edges in stationary blocks. To avoid edge related artifacts, blocks that contain an edge are unconditionally coded. The coder which incorporates the proposed change detector was found, in simulations, to provide a substantial reduction in bit-rate, while maintaining the quality of the reconstructed sequences, in coding image sequences which contain large areas of coarse textured background.

In this work we study morphological methods to reduce the amount of redundant points in the Skeleton representation of images. The advantage of removing redundant points using morphological operations only lies in the computational efficiency of these operations, when implemented on parallel machines. We propose a classification of redundant points of the skeleton into several categories and apply this classification in the framework of a generic approach which we present for obtaining redundancy-reduced Skeletons. This approach is shown to yield morphological closed formulae for reduced Skeletons which have less redundant points than the ordinary Skeleton. The approach is also extended for reducing the redundancy in Multi-Structuring-Element Skeletons (MSES). Although complete removal of the redundancy has not been achieved yet for the general case, the generic approach is shown to provide a complete removal of skeleton points belonging to most of the redundancy categories, as well as a redundancy-free representation for a particular important case of the MSES.

A new approach for gray-level image coding using binary morphological operations on the image bit-planes is presented. This approach is based on a Geometric Sampling Theorem (GST), and on a modified morphological skeleton. The theorem, which is proved in this paper, states conditions for the reconstruction of the boundary of a continuous two level image from a unique subset of points of its skeleton representation. This set of points, referred to as essential points, is found to play an important role in the skeleton representation of discrete binary images as well. The modified morphological skeleton (MMS) uses an exponentially increasing in size structuring element. The computational advantage of this representation was previously reported. A new approach to its development is presented here, and its advantage in image coding is demonstrated. The coding scheme consists of the following steps: First, the image is preprocessed by an error-diffusion technique in order to reduce the number of bit-planes from 8 to 4 without significant quality degradation. The pixel values are subsequently converted to Gray-code. The bit-planes are represented by the MMS. Redundancy in this representation is reduced using an algorithm motivated by the GST. These reduced modified morphological skeletons are coded with an entropy coding scheme particularly devised for efficient skeleton coding. The possibility of the introduction of geometric errors to reduce the bit-rate is also discussed. Compression ratios of up to 11:1 were obtained for satellite images.

We present the results of several psychophysical experiments that we conducted to determine appropriate values for threshold parameters to be used for block classification in an image-sequence conditional-replenishment encoding algorithm. The algorithm partitions the image into square blocks of pixels and, following motion compensation, classifies them into two categories, "nonsmooth" and "nontextured" ones. Nontextured blocks are tested with criteria different from those used for testing nonsmooth blocks for deciding whether to replenish or to copy the block. Since the end product of such algorithms is judged by human observers, the objective of the experiments that we conducted was to obtain pertinent characteristics of the human visual system (HVS) for incorporating them into the threshold selection part of the encoding process. In three separate major experiments we obtained near-optimal values for three threshold parameters by testing the detectability of stationary as well as moving targets under various conditions. In applying our observations to conditional replenishment coding, target detectability will refer to the visibility of motion-compensated prediction error and a consequent need to replenish the corresponding part of the image. The three specific experiments involved (a) uniform targets against a uniform background of different intensity, (b) textured targets against a uniform background of the same intensity as the average target intensity, and (c) textured targets against a textured background with the same standard deviation but different average intensities. The experiments reveal how to adjust the values of the thresholds to suit local conditions and they give rise to an adaptive threshold modification technique, based on the HVS characteristics. This approach achieved significant reductions in the bit rate needed to encode image sequences, without affecting the perceived image quality.

Performance of a distributed video coding system depends, to a large extent, on the accuracy of joint source and side information distribution modeling. In this work we first examine a family of stationary joint distribution models. As one of our findings, we propose to use the double-Gamma model as an alternative to the widely adopted Laplace model, due to its superior performance. In addition, we suggest a new spatially adaptive model, which enables to follow the spatially varying joint statistics of the source and side information. We present two methods, class-based and neighborhood-based, for estimation of the spatially varying model parameters. We then show how the obtained pixel domain model can be used in the transform domain to facilitate utilization of frame spatial redundancy. Integration of the proposed models into a distributed video coding system resulted in improved performance.

Closed-form expressions for a new set of 3D rotation invariants that are linear, quadratic, and angular combinations of implicit polynomial (IP) coefficients are developed. Based on these invariants, we propose a 3D object recognition method that outperforms recognition based on IP fitting after pose estimation, and the MPEG-7 SSD technique.

High quality low footprint Concatenative Text-To-Speech (CTTS) synthesizers provide a persistent challenge in the field of speech processing. The spectral parameters representing the short speech segments used in the concatenation process constitute a large portion of the required memory. In this paper we propose to use a vectorial form of Polynomial Temporal Decomposition combined with jointly optimal segmentation and polynomial order selection in order to reduce the storage required for the spectral amplitude parameters by 50%, while preserving the perceptual quality of the obtained synthesized speech.

The goal of this work is to develop methods for the adjustment of video in standard definition resolution to the smaller resolution used in mobile devices. The naive solution is to scale each frame to the desired size. But, objects must be at a sufficient size to be recognized easily. Therefore, it was suggested in previous works to display only a Region Of Interest (ROI). In this work we focus on news broadcasting and interview scenes, so the ROI contains the speaker's face. At the beginning of every scene, the ROI is marked by an editor. We present a novel algorithm for tracking the ROI. The algorithm estimates the global motion caused by the camera and the local motion caused by the speaker's movements. We estimate the motion parameters using horizontal and vertical projections. The slice-projection theorem and the Mellin transform are used for reducing complexity.

In this paper, we address the problem of unsupervised detection of anomalies in hyperspectral images. Our proposed method is based on a novel statistical background modeling approach that combines local and global approaches. The local-global background model has the ability to adapt to all nuances of the background process like local approaches but avoids over-fitting due to a too high number of degrees of freedom, which produces a high false alarm rate. This is done by constraining the local background models to be interrelated. The results strongly prove the effectiveness of the proposed algorithm. We experimentally show that our localglobal algorithm performs better than several other global or local anomaly detection techniques, such as the well known RX or its Gaussian Mixture version (GMRX).

Most image deblurring methods assume knowledge of the point spread function (PSF) causing the blur. In this work we address the problem of identifying the characterizing parameter of the PSF, which corresponds to motion or out-of-focus blur, from blurred and noisy images. The observation that the spectra of these blurring functions have periodic (or almost periodic) zeros is the basis of an already known blur identification method in the cepstral domain. However, this method is found to be highly sensitive to noise. In this paper we propose the following improvements on the above method: First, adding a preprocessing stage for noise reduction, using a modified spectral subtraction approach—with a median-complement filter to estimate the noise. Second, applying an adaptive, quefrency-varying, comb-like window (lifter) in the cepstral domain to enhance the blur parameter identification. The robustness of the proposed algorithm is demonstrated by its ability to identify the blur function parameters from noisy blurred images with signal-to-noise ratio down to 0 dB for motion blur and 3 dB for out-of-focus blur, as compared to 20 dB for the Original method.

In statistical TTS systems (STTS), speech features dynamics is modeled by firstand second-order feature frame differences, which, typically, do not satisfactorily represent frame to frame feature dynamics present in natural speech. The reduced dynamics results in over smoothing of speech features, often sounding as muffled synthesized speech. To improve feature dynamics a Global Variance approach has been suggested. However, it is computationally complex. We propose a different approach for modeling feature dynamics based on applying the DFT to the whole set of feature frames representing a phoneme. In the transform domain the inter-frame feature dynamics is then expressed in terms of inter-harmonic content, which can be modified to statistically match the dynamics of natural speech. To synthesize a whole utterance we propose a method for smoothly combining the enhanced-dynamics phonemes, which improves synthesized speech quality of STTS with similar complexity to conventional STTS.

In distributed video coding (DVC) systems the rate control is usually performed at the decoder through a feedback channel. However, a feedback channel incurs a delay and might be impractical for real-time applications. Typically, systems that suppress feedback use a fixed uniform quantization across the entire frame, regardless of its local characteristics. In this work a framework for intraframe rate allocation at the encoder side is suggested as a way to both suppress the feedback channel and adapt the quantization to the frame characteristics. The rate allocation utilizes an approximation model to the rate-distortion function for Laplacian data and correlation channel distributions. This model replaces an analytical solution given in integral form that can be evaluated only numerically. A practical DVC system that is based on this scheme is proposed. The use of intraframe rate allocation is indeed found to improve the performance of the feedback-less system.

This paper presents new properties of the discrete morphological skeleton representation of binary images, along with a novel coding scheme for lossless binary image compression that is based on these properties. Following a short review of the theoretical background, two sets of new properties of the discrete morphological skeleton representation of binary images are proved. The first one leads to the conclusion that only the radii of skeleton points belonging to a subset of the ultimate erosions are needed for perfect reconstruction. This corresponds to a lossless sampling of the quench function. The second set of new properties is related to deterministic prediction of skeletonal information in a progressive transmission scheme. Based on the new properties, a novel coding scheme for binary images is presented. The proposed scheme is suitable for progressive transmission and fast implementation. Computer simulations, also presented, show that the proposed coding scheme substantially improves the results obtained by previous skeleton-based coders, and performs better than classical coders, including run-length/Huffman, quadtree, and chain coders. For facsimile images, its performance can be placed between the modified read (MR) method (K=4) and modified modified read (MMR) method.

A system for bandwidth extension of telephone speech, aided by data embedding, is presented. The proposed system uses the transmitted analog narrowband speech signal as a carrier of the side information needed to carry out the bandwidth extension. The upper band of the wideband speech is reconstructed at the receiving end from two components: a synthetic wideband excitation signal, generated from the narrowband telephone speech and a wideband spectral envelope, parametrically represented and transmitted as embedded data in the telephone speech. We propose a novel data embedding scheme, in which the scalar Costa scheme is combined with an auditory masking model allowing high rate transparent embedding, while maintaining a low bit error rate. The signal is transformed to the frequency domain via the discrete Hartley transform (DHT) and is partitioned into subbands. Data is embedded in an adaptively chosen subset of subbands by modifying the DHT coefficients. In our simulations, high quality wideband speech was obtained from speech transmitted over a telephone line (characterized by spectral magnitude distortion, dispersion, and noise), in which side information data is transparently embedded at the rate of 600 information bits/second and with a bit error rate of approximately . In a listening test, the reconstructed wideband speech was preferred (at different degrees) over conventional telephone speech in of the test utterances.

Show-Through is a common occurrence when scanning duplex printed documents. The back-side printing shows through the paper, contaminating the front side image. Previous work modeled the problem as a non-linear convolutive mixture of images and offered solutions based on decorrelation. In this work we propose a cleaning process based on a Blind Source Separation approach. We define a cost function incorporating the non-linear mixing model in a mean-squared error term, along with a regularization term based on Total-Variation. We propose a location dependent regularization tradeoff, preserving image edges while removing show-through edges. The images and mixing parameters are estimated using an alternating minimization process, with each stage using only convex optimization methods. The resulting images exhibit significantly lower show-through, both visibly and in objective measures.

A common approach for video transrating (bit rate reduction) is to requantize the transform coefficients. Optimal requantization aims to find a set of new step-sizes that achieve the target bit rate while introducing minimal distortion. Since the state of the art H.264 standard coder constrains requantization by limiting the amount of change in the quantization step-size from one macroblock to the next, the common Lagrangian optimization approach cannot be applied. We propose a solution to this dependency problem by extending each Lagrangian iteration with a constrained dynamic programming stage. Further, in order to reduce the computational load of evaluating the rate and distortion at each macroblock for multiple step-sizes, we suggest analytic models that can be applied for this purpose. The developed models are suitable for requantization and are matched to the context-adaptive entropy coding used in H.264. The proposed algorithm performs the requantization in the compressed domain and currently supports inter coded frames only. It reduces the run-time by a factor of 4, as compared to the full exhaustive optimization, and achieves up to 1[dB] gain in PSNR, as compared to a simple one-pass algorithm.

Packet loss in internet audio streaming applications is a well-known and difficult problem. Unless concealed, each such loss produces an annoying disturbance. In this work we present a new algorithm for audio packet loss concealment, designed for MPEG-Audio streaming, based only on the data available at the receiver. The proposed algorithm reconstructs the missing data in the DSTFT (Discrete Short-Time Fourier-Transform) domain using either the GAPES (Gapped-data Amplitude and Phase Estimation) or the MAPES (Missing-data Amplitude and Phase Estimation) algorithm. The algorithm was implemented on an MP3 coder but is also suitable for MPEG-2/4 AAC. Examined subjectively by a group of listeners, the proposed algorithm performs better than other previously reported methods, even at high loss rates.

A system for converting Hebrew text into phonetic text is presented. The system applies speech recognition techniques for identifying vowels in spoken text and inserts these vowels in the original vowelless Hebrew text. The original Hebrew text is presented to the user on his computer terminal. The user then reads the text into a microphone and the system analyzes the input voice and converts each word into a string of Linear Prediction coefficients (LPC) vectors which are then compared to vectors stored in a set of codebooks (one codebook per vowel) to find the best match. The identified vowels and their location are then input to a Dynamic-Time-Warping algorithm which matches the acoustic information with the text in order to correctly position the vowels in each word using Hebrew syntax rules. The system could be used in preparing text for automatic, Hebrew Text-To-Speech ( TTS ) systems and in computer aided linguistic studies.

It is well known that wider bandwidth speech is preferred both for quality and intelligibility. In this paper we describe a new, low complexity method for creating wider bandwidth speech from clean telephone bandwidth speech. In addition, we describe a packet loss concealment technique that has been standardized for ITU-T Rec. G.711 64 kb/s PCM and is applicable to wider bandwidth speech as well. Together, these two techniques address two of the primary issues with Voice over IP-how to provide greater fidelity to customers and how to overcome packet losses when they do occur.

A basic reason for the loss of quality in LPC synthesized speech is the spectral mismatch between the LPC model and the analyzed speech. Since the residual signal contains the spectral information not extracted by the LPC analyzer, existing methods for improving the synthesized speech quality are based on encoding the residual signal or reducing its information content by improved pole-zero models. In this work we study an approach in which the spectral envelope of the residual signal is divided into three main components which are separately represented to provide an overall more efficient representation of the residual signal. The three spectral components are the antiresonances due to nasalized sounds, fixed or slowly varying spectral contributions due to input spectral shaping, and the remaining spectral matching error. An analysis-synthesis system based on the cepstral representation of the residual signal is developed and is shown to provide a particularly convenient framework for separating and representing by zeroes and cepstral residuals the different spectral components. Preliminary results of the study are presented on the basis of simulations with telephone bandwidth speech.

In this presentation the stable and optimal operation of the adaptive line enhancer (ALE) is considered. By improving the estimate of the steady-state mean square error (MSE), a tighter stability constraint is obtained, as well as more accurate expressions for the SNR gain attained by the ALE when filtering sinusoidal signals in white noise. Since the LMS algorithm used for adapting the ALE weights aims at minimizing the MSE, and not at maximizing the output SNR, the proper choice of the algorithm's parameters for maximizing the SNR gain is considered. In particular, it is shown that for a given step-size parameter µ (which satisfies the stability constraint) there exists an optimal number of weights which maximizes the SNR gain. Computer simulations verify the analytical results.

Because of speech signals nonstationarity, usual comb filtering of noisy speech signals results only in a modest improvement in signal to noise ratio, and only in a small perceptual reduction of structured noise or interference. A generalized comb filtering technique, which applies a time-varying weighting to each pitch period, is mathematically analyzed and shown to be capable of breaking up the noise structure, in addition to comb filtering. This is found to provide a meaningful perceptual improvement when the noise or interference are structured. The mathematical analysis is facilitated by using a polyphase network model of the generalized comb filter. Design constraints and rules are developed and several filter families are proposed. Computer simulation results are discussed.

An algebraic representation of the discrete short-time Fourier transform (DSTFT) is presented for the case in which the analysis window length N equals the transform block size M . This representation allows the application of algebraic tools for determining an optimal synthesis system which minimizes the mean square error between a given modified DSTFT (which is not necessarily a valid DSTFT sequence) and the DSTFT of the synthesized signal. If no modification is applied, the result is a unity analysis-synthesis system for any given time update R of the sliding analysis window (provided that R\leqM ). It is shown that the optimal synthesis system can be implemented by the well known weighted overlap-add (WOLA) method using an optimal synthesis window. The algebraic approach enables the extension of some recent results and the relaxation of a constraint on the analysis window. The proposed approach is found also to have a potential for solving the synthesis problem for the more general case of N>M .

A speech enhancement system which utilizes an optimal (in the minimum mean square error sense) short-time spectral amplitude estimator is described. The derivation of the optimal estimator is based on modeling speech as a quasi-periodic signal, and on applying spectral decomposition. The optimal spectral amplitude estimator and a recently developed vector spectral subtraction amplitude estimator, are round to be nearly equivalent. The optimal spectral amplitude estimator coincides with a Wiener spectral amplitude estimator at high signal to noise ratio (SNR) values, and is found to be superior to it at low SNR values. The enhanced speech obtained by using the proposed system, is less spectrally distorted, although contains some more residual noise, than the enhanced speech obtained by using the Wiener spectral amplitude estimator, in the same system. In addition, it is free of the "musical noise" characteristic to the spectral subtraction algorithm. Both systems, the proposed one and spectral subtraction, have approximately the same complexity.

The cepstral representation of the LPC residual signal was found in an earlier study to provide a convenient framework for efficiently representing the main spectral components of the residual signal. The analysis-synthesis system developed uses in general poles, zeros and cepstral residual terms to better represent the input speech signal spectrum and achieves improved synthesized speech quality. In this work we consider the quantization of the system parameters and its effect on the system performance in terms of spectral mismatch. The results are used to develop a cepstral residual vocoder system for 4.8 kbps transmission of speech. The system is based on an 8-pole LPC model and the cepstral representation of the residual signal. It applies optimal uniform quantization to each term of the cepstral residual and obtains most of the attainable spectral matching at this useful rate. The study is based on computer simulations with telephone bandwidth speech.