Search

This letter proposes a fast implementation of the regularity-constrained discrete sine transform (R-DST). The original DST \textit{leaks} the lowest frequency (DC: direct current) components of signals into high frequency (AC: alternating current) subbands. This property is not desired in many applications, particularly image processing, since most of the frequency components in natural images concentrate in DC subband. The characteristic of filter banks whereby they do not leak DC components into the AC subbands is called \textit{regularity}. While an R-DST has been proposed, it has no fast implementation because of the singular value decomposition (SVD) in its internal algorithm. In contrast, the proposed regularity-constrained fast sine transform (R-FST) is obtained by just appending a regularity constraint matrix as a postprocessing of the original DST. When the DST size is $M\times M$ ($M=2^\ell$, $\ell\in\mathbb{N}_{\geq 1}$), the regularity constraint matrix is constructed from only $M/2-1$ rotation matrices with the angles derived from the output of the DST for the constant-valued signal (i.e., the DC signal). Since it does not require SVD, the computation is simpler and faster than the R-DST while keeping all of its beneficial properties. An image processing example shows that the R-FST has fine frequency selectivity with no DC leakage and higher coding gain than the original DST. Also, in the case of $M=8$, the R-FST saved approximately $0.126$ seconds in a 2-D transformation of $512\times 512$ signals compared with the R-DST because of fewer extra operations.

This paper proposes an epigraphical relaxation (ERx) technique for non-proximable mixed norm minimization. Mixed norm regularization methods play a central role in signal reconstruction and processing, where their optimization relies on the fact that the proximity operators of the mixed norms can be computed efficiently. To bring out the power of regularization, sophisticated layered modeling of mixed norms that can capture inherent signal structure is a key ingredient, but the proximity operator of such a mixed norm is often unavailable (non-proximable). Our ERx decouples a layered non-proximable mixed norm into a norm and multiple epigraphical constraints. This enables us to handle a wide range of non-proximable mixed norms in optimization, as long as both the proximal operator of the outermost norm and the projection onto each epigraphical constraint are efficiently computable. Moreover, under mild conditions, we prove that ERx does not change the minimizer of the original problem despite relaxing equality constraints into inequality ones. We also develop new regularizers based on ERx: one is decorrelated structure-tensor total variation for color image restoration, and the other is amplitude-spectrum nuclear norm for low-rank amplitude recovery. We examine the power of these regularizers through experiments, which illustrates the utility of ERx., accepted to IEEE Transactions on Signal Processing

A limited number of types of sound event occur in an acoustic scene and some sound events tend to co-occur in the scene; for example, the sound events "dishes" and "glass jingling" are likely to co-occur in the acoustic scene "cooking". In this paper, we propose a method of sound event detection using graph Laplacian regularization with sound event co-occurrence taken into account. In the proposed method, the occurrences of sound events are expressed as a graph whose nodes indicate the frequencies of event occurrence and whose edges indicate the sound event co-occurrences. This graph representation is then utilized for the model training of sound event detection, which is optimized under an objective function with a regularization term considering the graph structure of sound event occurrence and co-occurrence. Evaluation experiments using the TUT Sound Events 2016 and 2017 detasets, and the TUT Acoustic Scenes 2016 dataset show that the proposed method improves the performance of sound event detection by 7.9 percentage points compared with the conventional CNN-BiGRU-based detection method in terms of the segment-based F1 score. In particular, the experimental results indicate that the proposed method enables the detection of co-occurring sound events more accurately than the conventional method., Accepted to IEICE Transactions on Information and Systems

A raw image captured by a color filter array (CFA), such as a Bayer pattern, is usually compressed after demosaicing with some processings (denoising, deblurring, tone-mapping, and so on). However, since photographers, designers, and high-end users prefer to work with the raw image sampled by CFA (referred to as “raw image”) directly, a raw image should be compressed before demosaicing. For effective raw image compression, this study introduces variable macropixel spectral-spatial transforms (VMSSTs), that can successfully decorrelate not only Bayer raw images but any other pure-color (RGB) ones. The proposed VMSSTs are designed by the following two steps: 1) intra-color decorrelation and 2) inter-color decorrelation. In lossless compression with JPEG 2000, compared with methods which do not use transforms, the VMSSTs reduced the average bitrates of three types of CFAs: from approximately 0.09 to 0.12 bpp for the modified Bayer CFA, from 0.25 to 0.65 bpp for the diagonal stripe CFA, and from 0.33 to 0.70 bpp for the Fujifilm X-Trans CFA due to their high color decorrelation efficiency. In addition, in lossy compression with JPEG 2000, compared with a rearranged method, the VMSSTs improved the average bitrates of the Bjontegaard delta by around 3.97%, 14.95%, and 18.65% for each CFA model, respectively. Although a data-dependent adaptive transformation, the Karhunen-Loeve transform (KLT), showed the best performance in lossy compression, the introduced VMSSTs have shown performances comparable to those of the KLT in lossless compression, despite their simple structures.

Block frames called directional analytic discrete cosine frames (DADCFs) are proposed for sparse image representation. In contrast to conventional overlapped frames, the proposed DADCFs require a reduced amount of 1) computational complexity, 2) memory usage, and 3) global memory access. These characteristics are strongly required for current high-resolution image processing. Specifically, we propose two DADCFs based on discrete cosine transform (DCT) and discrete sine transform (DST). The first DADCF is constructed from parallel separable transforms of DCT and DST, where the DST is permuted by row. The second DADCF is also designed based on DCT and DST, while the DST is customized to have no DC leakage property which is a desirable property for image processing. Both DADCFs have rich directional selectivity with slightly different characteristics each other and they can be implemented as non-overlapping block-based transforms, especially they form Parseval block frames with low transform redundancy. We perform experiments to evaluate our DADCFs and compare them with conventional directional block transforms in image recovery.

We present a novel image boundary extension and mean value extension (MVE) for directional lapped transforms, particularly cosine–sine modulated lapped transforms (CSMLTs). Lapped transforms are usually used with an extension technique, such as periodic extension (PE) or symmetric extension (SE), for nonexpansive convolution at signal boundaries. When directional textures (oblique lines and curves) appear at the 2D signal (image) boundaries, both PE and SE produce directional discontinuities, which degrade the sparsity of the transformed coefficients, especially in the case of directional lapped transforms. MVE reduces the discontinuities for directional textures better than PE or SE do; it thus improves the efficiency of the sparse representation based on directional lapped transforms. Moreover, to reduce computational costs compared with those of directional lapped transforms, we introduce new directional block transforms called cosine–sine modulated block transforms. These new transforms are derived from a minimum tile processing (tiling) of $M$ -band CSMLTs with $2M$ filter lengths and nonexpansive convolutions. The resulting directional block transforms, particularly in the case of the MVE, have richer directional selectivity and have better performance compared with a discrete Fourier transform as shown in experiments.

This paper proposes an epigraphical reformulation (ER) technique for non-proximable mixed norm regularization. Various regularization methods using mixed norms have been proposed, where their optimization relies on efficient computation of the proximity operator of the mixed norms. Although the sophisticated design of mixed norms significantly improves the performance of regularization, the proximity operator of such a mixed norm is often unavailable. Our ER decouples a non-proximable mixed norm function into a proximable norm and epigraphical constraints. Thus, it can handle a wide range of non-proximable mixed norms as long as the proximal operator of the outermost norm, and the projection onto the epigraphical constraints can be efficiently computed. Moreover, we prove that our ER does not change the minimizer of the original problem despite using a certain inequality approximation. We also provide a new structure-tensor-based regularization as an application of our framework, which illustrates the utility of ER.

The types of sound events that occur in a situation are limited, and some sound events are likely to co-occur; for instance, ``dishes'' and ``glass jingling.'' In this paper, we propose a technique of sound event detection utilizing graph Laplacian regularization taking the sound event co-occurrence into account. In the proposed method, sound event occurrences are represented as a graph whose nodes indicate the frequency of event occurrence and whose edges indicate the co-occurrence of sound events. This graph representation is then utilized for sound event modeling, which is optimized under an objective function with a regularization term considering the graph structure. Experimental results obtained using TUT Sound Events 2016 development, 2017 development, and TUT Acoustic Scenes 2016 development indicate that the proposed method improves the detection performance of sound events by 7.9 percentage points compared to that of the conventional CNN-BiGRU-based method in terms of the segment-based F1-score. Moreover, the results show that the proposed method can detect co-occurring sound events more accurately than the conventional method., Accepted to ICASSP 2019

This paper proposes multiscale structure-tensor total variation (MSTV) for image recovery. Gradient vectors in local patches usually have similar directions, and thus each local gradient matrix (the set of gradient vectors) tends to be low rank. STV introduces this property by calculating the sum of nuclear norms from all the 10-cal gradient matrices over the input image. By STV regularization, fine textures are recovered efficiently. However, since STV only considers differences of vertically and horizontally adjacent pixels, if neighboring samples are not reliable due to severe degradation, a latent image cannot be recovered efficiently. In this work, we assume that, for any two target pixels in a local patch, two vectors consisting of multiple differences not only between each target and adjacent pixels but also each target and further distant pixels exhibit a similar direction. According to this assumption, our MSTV firstly applies wavelet-based multiscale decomposition to vertical/horizontal gradient vectors and then evaluates the sum of nuclear norms of all the local wavelet coefficients. Experimental results show that the MSTV improves both numerical reconstruction error and subjective visual quality, compared with the conventional STV.

This paper presents M-channel ( $$M=2^{N}$$ , $$N\in \mathbb {N}$$ , $$N\ge 1$$ ) multiplierless lifting-based (ML-) fast X transforms (FXTs), where X $$=$$ F (Fourier), C (cosine), S (sine), and H (Hartley), i.e., FFT, FCT, FST, and FHT, derived from FHT factorization as way of lowering the cost of signal (image) processing. The basic forms of ML-FXTs are described. Then, they are customized for efficient image processing. The customized ML-FFT has a real-valued calculation followed by a complex-valued one. The ML-FCT customization for a block size of 8, which is a typical size for image coding, further reduces computational costs. We produce two customized ML-FCTs for lossy and lossless image coding. Numerical simulations show that ML-FFT and ML-FCTs perform comparably to the conventional methods in spite of having fewer operations.

This paper proposes a novel single-channel audio source separation based on graph-regularized nonnegative matrix factorization (NMF) taking harmonic frequency structure of each instrument into account. Since the original NMF, which is regarded as unsupervised learning, cannot readily identify the corresponding basis matrix for each target source, supervised NMFs (SNMFs) using given basis matrices learned from training sources have been extensively studied. Although SNMFs usually separate a mixed source better than NMF, the performance is degraded when training sources different from the observed source. The proposed SNMF does not use learned basis matrices but uses learned graph Laplacian matrices characterizing a harmonic frequency structure of training sources for regularization. Even if training sources are different from target sources, the graph structures from observed and training sources are more correlated, thus, as experimental results show, it can separate more robustly.

This paper proposes a new optimization framework for the joint optimization of sparsity and filter order (JOSFO) for FIR filter design. Since the cost function for JOSFO involves $\ell _{0}$ and non-uniform overlapped group $\ell _{0}$ norms, which are not convex, a global optimal solution is difficult to obtain. To find an approximate solution of the non-convex problem, existing approaches repeat the following steps: 1) approximate the cost function; 2) find candidates of zero coefficients by minimizing the cost function; and 3) set them to zero. On the other hand, this paper directly solves the optimization problem, without any approximation to the cost function, by using the alternating direction method of multipliers with the pseudo-proximity operators of $\ell _{0}$ and non-uniform non-overlapped group $\ell _{0}$ norms. Experimental results show that resulting filters designed by the proposed method have sparser coefficients and lower orders, while satisfying filter specifications, such as an error from a desired frequency response.

This paper proposes multiscale directional transforms (MDTs) based on cosine-sine modulated filter banks (CSMFBs). Sparse image representation by directional transforms is necessary for image analysis and processing tasks and has been extensively studied. Conventionally, cosine-sine modulated filter banks (CSMFBs) have been proposed as one of separable directional transforms (SepDTs). Their computational cost is much lower than non-SepDTs, and they can work better than other SepDTs, e.g., dual-tree complex wavelet transforms (DTCWTs) in image processing applications. One drawback of CSMFBs is a lack of multiscale directional selectivity, i.e., it cannot provide multiple scale directional atoms as in the DTCWT frame, and thus flexible image representation cannot be achieved. In this work, we show a design method of multiscale CSMFBs by extending modulated lapped transforms, which are a subclass of CSMFBs. We confirm its effectiveness in nonlinear approximation and image denoising as a practical application.

This paper proposes a novel image enhancement method for underwater images based on discrete cosine eigenbasis transformation (DCET). Since underwater images have a global strong dominant color, their colorfulness and contrast are often degraded. Typical color correction methods for natural images, i.e., computational color constancy, achromatize the colored illuminant of input images by dividing with an estimated illuminant color. However, this procedure produces unwanted color artifacts for underwater images. To solve this problem, we introduce a novel assumption that achromatic illuminant images should have the DCT basis vectors as their principal component vectors, which is termed as discrete cosine eigenbasis (DCE). According to the assumption, we achromatize underwater images by using a DCET that transforms the input image to the images having the DCE. By incorporating post image enhancement techniques, the proposed method provides sharper and brighter visual quality than conventional color correction and image enhancement methods.

Directional block transforms (DBTs), such as discrete Fourier transforms, are basically less efficient for sparse image representation than directional overlapped transforms, such as curvelet and contourlet, but have advantages in practical computation, such as less computational cost, less amount of memory usage to be used, and parallel processing. In order to realize efficient DBTs, this paper proposes directional discrete cosine transforms (DDCTs) by using discrete cosine and sine transforms. The resulting transforms provide richer directional orientations of atoms than conventional DBTs, and thus they are expected to be more efficient for image analysis and processing. In experiments, we evaluate DDCTs with conventional DBTs in image recovery by a convex optimization.

Complex-valued conjugate-symmetric Hadamard transforms ( $\BBC $ -CSHT) are variants of complex Hadamard transforms and found applications in signal processing. In addition, their real-valued transform counterparts ( $\BBR $ -CSHTs) perform comparably with Hadamard transforms (HTs) despite their lower computational complexity. Closed-form factorizations of $\BBC $ -CSHTs and $\BBR $ -CSHTs have recently been proposed to make calculations more efficient. However, there is still room to find effective and general factorizations. This paper presents a simple closed-form complete factorization of $\BBC $ -CSHTs based on that of $\BBR $ -CSHTs. The proposed factorization can be applied to both $\BBC $ - and $\BBR $ -CSHTs with one factorization and it provides several benefits: 1) It can save total implementation costs for both $\BBC $ -CSHTs and $\BBR $ -CSHTs; 2) the generalized $\BBR $ -CSHT factorization significantly reduces its computational cost; 3) memory-saved local orientation detection of images can be achieved; 4) a fast direction-aware transform can be attained; 5) it clarifies that $\BBC $ - and $\BBR $ -CSHTs are closely related to common block transforms, such as the discrete Fourier transform (DFT), binDCT, and HT; and 6) it achieves a new integer complex-valued transform, which can approximate the DFT better than the original $\BBC $ -CSHT. The image orientation estimation and performance in image coding of our $\BBR $ -CSHTs were evaluated through examples of practical applications based on the proposed factorization.