Search

In recent years, eye-tracking technology has gained considerable attention, facilitating analysis of gaze behavior and human visual attention. However, eye-tracking analysis often requires manual annotation on the objects being gazed upon, making quantitative data analysis a difficult and time-consuming process. This thesis explores the area of object detection and object tracking applied on scene camera footage from mobile eye-tracking glasses. We have evaluated the performance of state-of-the-art object detectors and trackers, resulting in an automated pipeline specialized at detecting and tracking objects in scene videos. Motion blur constitutes a significant challenge in moving cameras, complicating tasks such as object detection and tracking. To address this, we explored two approaches. The first involved retraining object detection models on datasets with augmented motion-blurred images, while the second one involved preprocessing the video frames with deblurring techniques. The findings of our research contributes with insights into efficient approaches to optimally detect and track objects in scene camera footage from eye-tracking glasses. Out of the technologies we tested, we found that motion deblurring using DeblurGAN-v2, along with a DINO object detector combined with the StrongSORT tracker, achieved the highest accuracies. Furthermore, we present an annotated dataset consisting of frames from recordings with eye-tracking glasses, that can be utilized for evaluating object detection and tracking performance.

In recent years, eye-tracking technology has gained considerable attention, facilitating analysis of gaze behavior and human visual attention. However, eye-tracking analysis often requires manual annotation on the objects being gazed upon, making quantitative data analysis a difficult and time-consuming process. This thesis explores the area of object detection and object tracking applied on scene camera footage from mobile eye-tracking glasses. We have evaluated the performance of state-of-the-art object detectors and trackers, resulting in an automated pipeline specialized at detecting and tracking objects in scene videos. Motion blur constitutes a significant challenge in moving cameras, complicating tasks such as object detection and tracking. To address this, we explored two approaches. The first involved retraining object detection models on datasets with augmented motion-blurred images, while the second one involved preprocessing the video frames with deblurring techniques. The findings of our research contributes with insights into efficient approaches to optimally detect and track objects in scene camera footage from eye-tracking glasses. Out of the technologies we tested, we found that motion deblurring using DeblurGAN-v2, along with a DINO object detector combined with the StrongSORT tracker, achieved the highest accuracies. Furthermore, we present an annotated dataset consisting of frames from recordings with eye-tracking glasses, that can be utilized for evaluating object detection and tracking performance.

Video restoration concerns the recovery of a clean video sequence starting from its degraded version. Different video restoration tasks exist, including denoising, deblurring, super-resolution, and reduction of compression artifacts. In this paper, we provide a comprehensive review of the main features of existing video restoration methods based on deep learning. We focus our attention on the main architectural components, strategies for motion handling, and loss functions. We analyze the standard benchmark datasets and use them to summarize the performance of video restoration methods, both in terms of effectiveness and efficiency. In conclusion, the main challenges and future research directions in video restoration using deep learning are highlighted.

The increasing need of restoring high-resolution hyper-spectral (HS) images is determining a growing reliance on computer vision-based processing to enhance the clarity of the image content. HS images can, in fact, suffer from degradation effects or artefacts caused by instrument limitations. This article focuses on a procedure aimed at reducing the degradation effects, frequency-dependent blur and noise, in Terahertz time-domain spectroscopy (THz-TDS) images in reflection geometry. It describes the application of a joint deblurring and denoising approach that had been previously proved to be effective for the restoration of THz-TDS images in transmission geometry, but that had never been tested in reflection modality. This mode is often the only one that can be effectively used in most cases, for example, when analyzing objects that are either opaque in the THz range, or that cannot be displaced from their location (e.g., museums), such as those of cultural interest. Compared to transmission mode, reflection geometry introduces, however, further distortion to THz data, usually neglected in existing literature. In this work, we successfully implement image deblurring and denoising of both uniform-shape samples (a contemporary 1 Euro cent coin and an inlaid pendant) and samples with the uneven reliefs and corrosion products on the surface, which make the analysis of the object particularly complex (an ancient Roman silver coin). The study demonstrates the ability of image processing to restore data in the 0.25–6 THz range, spanning over more than four octaves, and providing the foundation for future analytical approaches of cultural heritage using the far-infrared spectrum still not sufficiently investigated in the literature.

Computer algebra is applied to biomedical computer vision. Specifically certain biomedical images resulting from a case of rhinocerebral mucormysocis in a diabetic boy are analyzed using techniques in computational geometry and in algebraic-geometric topology. We apply convolution and deblurring via diffusion equation from the side of computational geometry and knot theory, graph theory and singular homology form the side of algebraic-geometric topology. Our strategy consists in to represent the biomedical images using algebraic structures in such way that the peculiarities of the images are represented using algebraic complexities. With our strategy we obtain an automatic procedure for the analysis and the diagnostic based on biomedical images. © 2010 Springer-Verlag Berlin Heidelberg.

See abstract., Various frequency and time-domain signal processing exercises, including guitar effects and fourier filtering, for Matlab implementation. Untried in actual pedagogical situation, but implemented by CB. This document is highest-level page only, to access embedded links go to http://www.cs.rochester.edu/u/brown/160

In this paper we address the problem of recovering an image from a sequence of distorted versions of it, where the distortion is caused by what is commonly referred to as ground-level turbulence. In mathematical terms, such distortion can be described as the cumulative effect of a blurring kernel and a time-dependent deformation of the image domain. We introduce a statistical dynamic model for the generation of turbulence based on linear dynamical systems (LDS). We expand the model to include the unknown image as part of the unobserved state and apply Kalman filtering to estimate such state. This operation yields a blurry image where the blurring kernel is effectively isoplanatic. Applying blind nonlocal Total Variation (NL-TV) deconvolution yields a sharp final result. © 2013 Springer Science+Business Media New York.

In this paper, we deal with the image deblurring problem in a completely new perspective by proposing separable kernel to represent the inherent properties of the camera and scene system. Specifically, we decompose a blur kernel into three individual descriptors (trajectory, intensity and point spread function) so that they can be optimized separately. To demonstrate the advantages, we extract one-pixel-width trajectories of blur kernels and propose a random perturbation algorithm to optimize them but still keeping their continuity. For many cases, where current deblurring approaches fall into local minimum, excellent deblurred results and correct blur kernels can be obtained by individually optimizing the kernel trajectories. Our work strongly suggests that more constraints and priors should be introduced to blur kernels in solving the deblurring problem because blur kernels have lower dimensions than images. © 2014 IEEE.

[ANGLÈS] Recent research on the image analysis field has been focused on using the redudancy of data present in a single image to solve various image processing problems. The analysis of multi-view scenarios has been set aside, although these environments present interesting geometrical properties that can be used to determine the location of similar regions among other images, which could enhance the results present in just the image itself. In this thesis we propose a geometrical and generalizable prior to efficiently exploit the redundancy of data in multi-view settings by determining suitable search regions with epipolar lines. A search algorithm to effectively apply the mentioned prior is also presented, along with its performance to solve different image processing applications., [CASTELLÀ] Recientes investigaciones en el análisis de imágenes optan por usar la redundancia de la información presente en una misma imagen para resolver diversos problemas de procesado. Los escenarios multivista han sido dejados de lado a pesar de presentar ciertas propiedades geométricas que permiten localizar regiones similares en diferentes imágenes, algo que podría mejorar los resultados obtenidos mediante el análisis de una sola imagen. En este proyecto se propone un principio geométrico y generalizable para explotar eficientemente la redundancia en entornos multivista determinando regiones de búsqueda apropiadas mediante líneas epipolares. También se detalla un algoritmo de búsqueda que usa dicho principio, así como resultados mostrando la capacidad de nuestro sistema para resolver diferentes aplicaciones de procesado de imagen., [CATALÀ] Les recerques més recents en l'anàlisi d'imatges opten per emprar la redundància de l'informació present en una mateixa imatge per a resoldre diversos problemes de processament. Els entorns multivista han sigut deixats de banda tot i que presenten certes propietats geomètriques que permeten localitzar regions similars entre diferents imatges, cosa que pot millorar els resultats obtinguts mitjançant 'anàlisi d'una sola imatge. En aquest projecte es proposa un principi geomètric generalitzable per a emprar eficientment la redundància d'entorns multivista determinant regions de cerca apropiades mitjançant linies epipolars. També s'exposa un algorisme de cerca que utilitza aquest principi, així com resultats que mostren la capacitat del nostre sistea per a resoldre diferents aplicacions de processament d'imatge., Desarrollo de un algoritmo de búsqueda eficiente y "prior" para entornos multivista con diversas aplicaciones.

In this paper, we deal with the image deblurring problem in a completely new perspective by proposing separable kernel to represent the inherent properties of the camera and scene system. Specifically, we decompose a blur kernel into three individual descriptors (trajectory, intensity and point spread function) so that they can be optimized separately. To demonstrate the advantages, we extract one-pixel-width trajectories of blur kernels and propose a random perturbation algorithm to optimize them but still keeping their continuity. For many cases, where current deblurring approaches fall into local minimum, excellent deblurred results and correct blur kernels can be obtained by individually optimizing the kernel trajectories. Our work strongly suggests that more constraints and priors should be introduced to blur kernels in solving the deblurring problem because blur kernels have lower dimensions than images. © 2014 IEEE.

In this paper we address the problem of recovering an image from a sequence of distorted versions of it, where the distortion is caused by what is commonly referred to as ground-level turbulence. In mathematical terms, such distortion can be described as the cumulative effect of a blurring kernel and a time-dependent deformation of the image domain. We introduce a statistical dynamic model for the generation of turbulence based on linear dynamical systems (LDS). We expand the model to include the unknown image as part of the unobserved state and apply Kalman filtering to estimate such state. This operation yields a blurry image where the blurring kernel is effectively isoplanatic. Applying blind nonlocal Total Variation (NL-TV) deconvolution yields a sharp final result. © 2013 Springer Science+Business Media New York.

Desarrollo de un algoritmo de búsqueda eficiente y "prior" para entornos multivista con diversas aplicaciones., [ANGLÈS] Recent research on the image analysis field has been focused on using the redudancy of data present in a single image to solve various image processing problems. The analysis of multi-view scenarios has been set aside, although these environments present interesting geometrical properties that can be used to determine the location of similar regions among other images, which could enhance the results present in just the image itself. In this thesis we propose a geometrical and generalizable prior to efficiently exploit the redundancy of data in multi-view settings by determining suitable search regions with epipolar lines. A search algorithm to effectively apply the mentioned prior is also presented, along with its performance to solve different image processing applications., [CASTELLÀ] Recientes investigaciones en el análisis de imágenes optan por usar la redundancia de la información presente en una misma imagen para resolver diversos problemas de procesado. Los escenarios multivista han sido dejados de lado a pesar de presentar ciertas propiedades geométricas que permiten localizar regiones similares en diferentes imágenes, algo que podría mejorar los resultados obtenidos mediante el análisis de una sola imagen. En este proyecto se propone un principio geométrico y generalizable para explotar eficientemente la redundancia en entornos multivista determinando regiones de búsqueda apropiadas mediante líneas epipolares. También se detalla un algoritmo de búsqueda que usa dicho principio, así como resultados mostrando la capacidad de nuestro sistema para resolver diferentes aplicaciones de procesado de imagen., [CATALÀ] Les recerques més recents en l'anàlisi d'imatges opten per emprar la redundància de l'informació present en una mateixa imatge per a resoldre diversos problemes de processament. Els entorns multivista han sigut deixats de banda tot i que presenten certes propietats geomètriques que permeten localitzar regions similars entre diferents imatges, cosa que pot millorar els resultats obtinguts mitjançant 'anàlisi d'una sola imatge. En aquest projecte es proposa un principi geomètric generalitzable per a emprar eficientment la redundància d'entorns multivista determinant regions de cerca apropiades mitjançant linies epipolars. També s'exposa un algorisme de cerca que utilitza aquest principi, així com resultats que mostren la capacitat del nostre sistea per a resoldre diferents aplicacions de processament d'imatge.

In this paper we propose a novel framework for fast exploitation of multi-view cues with applicability in different image processing problems. In order to bring our proposed framework into practice, an epipolar-constrained prior is presented, onto which a random search algorithm is proposed to find good matches among the different views of the same scene. This algorithm includes a generalization of the local coherency in 2D images for multi-view wide-baseline cases. Experimental results show that the geometrical constraint allows a faster initial convergence when finding good matches. We present some applications of the proposed framework on classical image processing problems., Peer Reviewed, Postprint (published version)

Retinal images often suffer from blurring which hinders disease diagnosis and progression assessment. The restoration of the images is carried out by means of blind deconvolution, but the success of the restoration depends on the correct estimation of the point-spread-function (PSF) that blurred the image. The restoration can be space-invariant or space-variant. Because a retinal image has regions without texture or sharp edges, the blind PSF estimation may fail. In this paper we propose a strategy for the correct assessment of PSF estimation in retinal images for restoration by means of space-invariant or space-invariant blind deconvolution. Our method is based on a decomposition in Zernike coefficients of the estimated PSFs to identify valid PSFs. This significantly improves the quality of the image restoration revealed by the increased visibility of small details like small blood vessels and by the lack of restoration artifacts., Postprint (published version)

In this paper, we deal with the image deblurring problem in a completely new perspective by proposing separable kernel to represent the inherent properties of the camera and scene system. Specifically, we decompose a blur kernel into three individual descriptors (trajectory, intensity and point spread function) so that they can be optimized separately. To demonstrate the advantages, we extract one-pixel-width trajectories of blur kernels and propose a random perturbation algorithm to optimize them but still keeping their continuity. For many cases, where current deblurring approaches fall into local minimum, excellent deblurred results and correct blur kernels can be obtained by individually optimizing the kernel trajectories. Our work strongly suggests that more constraints and priors should be introduced to blur kernels in solving the deblurring problem because blur kernels have lower dimensions than images. © 2014 IEEE.

Retinal images are essential clinical resources for the diagnosis of retinopathy and many other ocular diseases. Because of improper acquisition conditions or inherent optical aberrations in the eye, the images are often degraded with blur. In many common cases, the blur varies across the field of view. Most image deblurring algorithms assume a space-invariant blur, which fails in the presence of space-variant (SV) blur. In this work, we propose an innovative strategy for the restoration of retinal images in which we consider the blur to be both unknown and SV. We model the blur by a linear operation interpreted as a convolution with a point-spread function (PSF) that changes with the position in the image. To achieve an artifact-free restoration, we propose a framework for a robust estimation of the SV PSF based on an eye-domain knowledge strategy. The restoration method was tested on artificially and naturally degraded retinal images. The results show an important enhancement, significant enough to leverage the images' clinical use. (C) 2014 Society of Photo-Optical Instrumentation Engineers (SPIE), Postprint (published version)

The U.S. Army Research Laboratory (ARL) Image Enhancement Test Bed was developed to evaluate various image processing research algorithms being developed at ARL. The Test Bed integrates these algorithms into a more real-world environment simulating Army-relevant scenarios. Some of the algorithms are made into client applications such as Super Resolution, Contrast, and Deblur, which can be accessed from an Image Processing Representational State Transfer (REST) Web service by a cell phone, laptop PC, Microsoft Surface Touch Table, as well as other devices. Other more central processing unit (CPU) intensive and time-constrained algorithms such as tracking algorithms run directly on the laptop PC or Touch Table Computer. This report briefly describes the image processing algorithms implemented in the ARL Image Enhancement Test Bed while citing the supporting research publications. The main purpose of this report is to provide a detailed description of the ARL Image Enhancement Test Bed Environment and graphical user interface (GUI), and serve as a user s manual.

In this report, we describe a fast deconvolution approach for color images that combines a sparse regularization cost on the magnitudes of gradients with constraints on their direction in color space. We form these color constraints in a way that allows retaining the computationally-efficient optimization strategy introduced in recent deconvolution methods based on half-quadratic splitting. The proposed algorithm is capable of handling a different blur kernel in each color channel, and is used for per-layer deconvolution in our paper: Depth and Deblurring from a Spectrally-varying Depth-of-Field [1]. A MATLAB implementation of this method is available at http://vision.seas.harvard. edu/ccap, and takes roughly 20 seconds to deconvolve a three-channel 1544 1028 color image, on a Linux-based Intel I-3 2.1GHz machine.

Postprint (published version)

In this paper, a new variational model for restoring blurred images with multiplicative noise is proposed. Based on the statistical property of the noise, a quadratic penalty function technique is utilized in order to obtain a strictly convex model under a mild condition, which guarantees the uniqueness of the solution and the stabilization of the algorithm. For solving the new convex variational model, a primal-dual algorithm is proposed, and its convergence is studied. The paper ends with a report on numerical tests for the simultaneous deblurring and denoising of images subject to multiplicative noise. A comparison with other methods is provided as well.

Super Resolution has been one of the main topics of research in Image Processing in the last twenty years. Different authors have presented a vast amount of different techniques and approaches. Nowadays reconstruction-­‐based Super Resolution works effectively for synthetic data but practical problems appear in real scenarios with objects moving with arbitrary speeds and directions within the scene. With our work, we present a review of the state-­‐of-­‐the-­‐art in reconstruction-­‐based Super Resolution, we make contributions to the involved stages (registration, reconstruction and restoration) and finally, we introduce some initial ideas for future work leading to effective Super Resolution even in sequences with highly complex motion., [ANGLÈS] Use of the inpainting technique to reconstruct an image after applying registration in the multi-frames input. Use of restoration methods to increment the resolution of the reconstructed image., [CASTELLÀ] Aplicación del método inpainting para reconstrucción después de haber obtenido el registro de las múltiples imágenes de entrada. Aplicación de técnicas de restauración para incrementar la resolución de la imagen reconstruida., [CATALÀ] Aplicació del mètode inpainting per reconstrucció després d'haver obtingut el registre de les múltiples imatges d'entrada. Aplicació de tècniques de restauració per incrementar la resolució de la imatge reconstruïda.

We propose here a class of restoration algorithms for color images, based upon the Mumford-Shah (MS) model and nonlocal image information. The Ambrosio-Tortorelli and Shah elliptic approximations are defined to work in a small local neighborhood, which are sufficient to denoise smooth regions with sharp boundaries. However, texture is nonlocal in nature and requires semilocal/non-local information for efficient image denoising and restoration. Inspired from recent works (nonlocal means of Buades, Coll, Morel, and nonlocal total variation of Gilboa, Osher), we extend the local Ambrosio-Tortorelli and Shah approximations to MS functional (MS) to novel nonlocal formulations, for better restoration of fine structures and texture. We present several applications of the proposed nonlocal MS regularizers in image processing such as color image denoising, color image deblurring in the presence of Gaussian or impulse noise, color image inpainting, color image super-resolution, and color filter array demosaicing. In all the applications, the proposed nonlocal regularizers produce superior results over the local ones, especially in image inpainting with large missing regions. We also prove several characterizations of minimizers based upon dual norm formulations.

We propose here a class of restoration algorithms for color images, based upon the Mumford-Shah (MS) model and nonlocal image information. The Ambrosio-Tortorelli and Shah elliptic approximations are defined to work in a small local neighborhood, which are sufficient to denoise smooth regions with sharp boundaries. However, texture is nonlocal in nature and requires semilocal/non-local information for efficient image denoising and restoration. Inspired from recent works (nonlocal means of Buades, Coll, Morel, and nonlocal total variation of Gilboa, Osher), we extend the local Ambrosio-Tortorelli and Shah approximations to MS functional (MS) to novel nonlocal formulations, for better restoration of fine structures and texture. We present several applications of the proposed nonlocal MS regularizers in image processing such as color image denoising, color image deblurring in the presence of Gaussian or impulse noise, color image inpainting, color image super-resolution, and color filter array demosaicing. In all the applications, the proposed nonlocal regularizers produce superior results over the local ones, especially in image inpainting with large missing regions. We also prove several characterizations of minimizers based upon dual norm formulations.

We propose here a class of restoration algorithms for color images, based upon the Mumford-Shah (MS) model and nonlocal image information. The Ambrosio-Tortorelli and Shah elliptic approximations are defined to work in a small local neighborhood, which are sufficient to denoise smooth regions with sharp boundaries. However, texture is nonlocal in nature and requires semilocal/non-local information for efficient image denoising and restoration. Inspired from recent works (nonlocal means of Buades, Coll, Morel, and nonlocal total variation of Gilboa, Osher), we extend the local Ambrosio-Tortorelli and Shah approximations to MS functional (MS) to novel nonlocal formulations, for better restoration of fine structures and texture. We present several applications of the proposed nonlocal MS regularizers in image processing such as color image denoising, color image deblurring in the presence of Gaussian or impulse noise, color image inpainting, color image super-resolution, and color filter array demosaicing. In all the applications, the proposed nonlocal regularizers produce superior results over the local ones, especially in image inpainting with large missing regions. We also prove several characterizations of minimizers based upon dual norm formulations.

An image representation framework based on structured sparse model selection is introduced in this work. The corresponding modeling dictionary is comprised of a family of learned orthogonal bases. For an image patch, a model is first selected from this dictionary through linear approximation in a best basis, and the signal estimation is then calculated with the selected model. The model selection leads to a guaranteed near optimal denoising estimator. The degree of freedom in the model selection is equal to the number of the bases, typically about 10 for natural images, and is significantly lower than with traditional overcomplete dictionary approaches, stabilizing the representation. For an image patch of size sq root N x sq root N, the computational complexity of the proposed framework is O(N2), typically 2 to 3 orders of magnitude faster than estimation in an overcomplete dictionary. The orthogonal bases are adapted to the image of interest and are computed with a simple and fast procedure. State-of-the-art results are shown in image denoising, deblurring, and inpainting., Sponsored in part by NGA, ARO, NSF, and NIH.

The continuous advancement in wireless communications technology demands new approaches to improving the capacity of existing radio links. The high data throughput required can be achieved by the complete utilization of space, time and polarization diversities inherent in any propagation environment. Among the different propagation scenarios, the indoor channels represent a particularly challenging problem given the number and complexity of interactions between the transmitted signal and the environment. This dissertation explores the interrelation between propagation physics and space-time-polarization diversity based on a novel high resolution channel sounding and reconstruction technique. First, a method to reconstruct the indoor complex channel response based on a limited set of samples and the elimination of the interference using deconvolution techniques is presented. Then, the results for the joint angle-of-arrival, delay characterization and depolarization of electromagnetic waves are presented. Finally, a novel approach to using depolarized multipath signals to boost the receiver signal-to-noise performance is presented. The current study shows that full utilization of the diversities of channel novel wireless systems can be proposed with significant improvement in capacity.

The continuous advancement in wireless communications technology demands new approaches to improving the capacity of existing radio links. The high data throughput required can be achieved by the complete utilization of space, time and polarization diversities inherent in any propagation environment. Among the different propagation scenarios, the indoor channels represent a particularly challenging problem given the number and complexity of interactions between the transmitted signal and the environment. This dissertation explores the interrelation between propagation physics and space-time-polarization diversity based on a novel high resolution channel sounding and reconstruction technique. First, a method to reconstruct the indoor complex channel response based on a limited set of samples and the elimination of the interference using deconvolution techniques is presented. Then, the results for the joint angle-of-arrival, delay characterization and depolarization of electromagnetic waves are presented. Finally, a novel approach to using depolarized multipath signals to boost the receiver signal-to-noise performance is presented. The current study shows that full utilization of the diversities of channel novel wireless systems can be proposed with significant improvement in capacity.

Image restoration based on an l1-data-fitting term and edge preserving total variation regularization is considered. The associated nonsmooth energy minimization problem is handled by utilizing Fenchel duality and dual regularization techniques. The latter guarantee uniqueness of the dual solution and an efficient way for reconstructing a primal solution, i.e., the restored image, from a dual solution. For solving the resulting primal-dual system, a semismooth Newton solver is proposed and its convergence is studied. The paper ends with a report on restoration results obtained by the new algorithm for salt-and-pepper or random-valued impulse noise including blurring. A comparison with other methods is provided as well.

Multiscale representation of a given image is the problem of constructing a family of images, where each image in this family represents a scaled version of the given image. This finds its motivation from biological vision studies. Using the hierarchical multiscale image representation proposed by Tadmor et. al. [32], an image is decomposed into sums of simpler `slices', which extract more refined information from the previous scales. This approach motivates us to propose a novel integro-differential equation (IDE), for a multiscale image representation. We examine various properties of this IDE. The advantage of formulating the IDE this way is that, although this IDE is motivated by variational approach, we no longer need to be associated with any minimization problem and can modify the IDE, suitable to our image processing needs. For example, we may need to find different scales in the image, while retaining or enhancing prominent edges, which may define boundaries of objects. We propose some edge preserving modifications to our IDE. One of the important problems in image processing is deblurring a blurred image. Images get blurred due to various reasons, such as unfocused camera lens, relative motion between the camera and the object pictured, etc. The blurring can be modeled with a continuous, linear operator. Recovering a clean image from a blurry image, is an ill-posed problem, which is solved using Tikhonov-like regularization. We propose a different IDE to solve the deblurring problem. We propose hierarchical multiscale scheme based on (BV; L1) decomposition, proposed by Chan, Esedoglu, Nikolova and Alliney [12, 25, 3]. We finally propose another hierarchical multiscale representation based on a novel weighted (BV;L1) decomposition.

Image restoration based on an l1-data-fitting term and edge preserving total variation regularization is considered. The associated nonsmooth energy minimization problem is handled by utilizing Fenchel duality and dual regularization techniques. The latter guarantee uniqueness of the dual solution and an efficient way for reconstructing a primal solution, i.e., the restored image, from a dual solution. For solving the resulting primal-dual system, a semismooth Newton solver is proposed and its convergence is studied. The paper ends with a report on restoration results obtained by the new algorithm for salt-and-pepper or random-valued impulse noise including blurring. A comparison with other methods is provided as well.

Many problems in image processing are addressed via the minimization of a cost functional. The most prominent optimization technique is the gradient descent, often used due to its simplicity and applicability where other techniques, e.g., those coming from discrete optimization, can not be used. Yet, gradient descent suffers from a slow convergence, and often to just local minima which highly depends on the condition number of the functional Hessian. Newton-type methods, on the other hand, are known to have a rapid, quadratic, convergence. In its classical form, the Newton method relies on the L2-type norm to define the descent direction. In this paper, we generalize and reformulate this very important optimization method by introducing a novel Newton method based on more general norms. This generalization opens up new possibilities in the extraction of the Newton step, including benefits such as mathematical stability and the incorporation of smoothness constraints. We first present the derivation of the modified Newton step in the calculus of variation framework needed for image processing. Then, we demonstrate the method with two common objective functionals: variational image deblurring and geodesic active contours for image segmentation. We show that in addition to the fast convergence, norms adapted to the problem at hand yield different and superior results.

The framework of kernel regression, a nonparametric estimation method, has been widely used in different guises for solving a variety of image processing problems including denoising and interpolation. In this paper, we extend the use of kernel regression for deblurring applications. Furthermore, we show that many of the popular image reconstruction techniques are special cases of the proposed framework. Simulation results confirm the effectiveness of our proposed methods., Sponsored in part by the National Science Foundation Science and Technology Center for Adaptive Optics under Grant no. AST-9876783. Presented at the Asilomar Conference on Signals, Systems, and Computers (40th) held in Pacific Grove, CA on 29 Oct-1 Nov 2006 and published in proceedings of the same.

This technical report investigates the characteristics of compression noise in images and motion imagery compressed by scalar quantization of the data's two- or three-dimensional wavelet transform coefficients. Such quantization noise is both experimentally and theoretically shown to be spatially varying in the pixel domain, with statistical correlations between the errors at the pixel locations. A quantization noise covariance matrix is presented that can find use in general restoration scenarios where the observed image or images have been compressed by scalar quantization of the data's wavelet coefficients. Several restoration examples, including de-blurring for the single-image case and temporal filtering for the motion-imagery case, are provided to demonstrate the quantization noise model's advantage over the common assumption of independent and identically distributed noise. (7 tables, 19 figures, 29 refs.), The original document contains color images.

This work extends the multiplicative visual model to include image texture as suggested by previous experiments linking a low resolution Fourier analysis with neurons in certain parts of the visual cortex. The new model takes image texture into account in the sense that weak texture is accentuated and strong, high contrast texture is attenuated. This model is then used as the basis for an improved image enhancement scheme and an unusually successful method for restoring blurred images. In addition, it is suggested how the model may provide new insights into the problem of finding a quantitatively correct image fidelity criterion. The structure of this model is described in relation to visual neurophysiology and examples are presented of images processed by the new techniques. The present research also shows how Land's 'retinex' can be implemented in a new way which allows the required computations to be carried out on a rectangular grid., Doctoral thesis. Original contains color plates: All DDC reproductions will be in black and white.